Ear return delay detection method and device, electronic equipment and storage medium

Abstract

The invention provides an ear return time delay detection method and device, electronic equipment and a storage medium, the method is used for detecting ear return time delay of an ear return system, the ear return system at least comprises an audio collector and an audio player, and first cache time delay is determined according to collection cache data volume in the audio collector; determining the time delay of the audio collector according to the first cache time delay and the hardware time delay of the audio collector; determining a second cache delay according to the playing cache data volume in the audio player; determining the time delay of the audio player according to the second cache time delay and the hardware time delay of the audio player; according to the time delay of the audio collector and the time delay of the audio player, determining the ear return time delay of the ear return system; by adopting the method provided by the invention, the cached data volume of the ear return system is utilized to carry out ear return delay detection, and external equipment does not need to be connected to carry out delay detection, so that the detection process of the ear return delay is simplified, and real-time calculation on the ear return delay of the ear return system is facilitated.

Description

Technical Field

[0001] The present application relates to the field of sound detection technology, and specifically to an ear return delay detection method, device, electronic device and storage medium. Background Art

[0002] In audio and video application scenarios, it is often necessary to listen to the audio through an earphone monitor. For example, when a singer is singing, he or she can use the earphone monitor to listen to his or her own voice and correct his or her pronunciation in time to avoid going out of tune. The delay of the earphone monitor is a major indicator of the earphone monitor function, which directly affects the normal use of the earphone monitor function.

[0003] At present, the commonly used method for detecting earphone feedback delay is to measure the waveform distance through an oscilloscope and calculate the earphone feedback delay based on the waveform distance. The detection process requires external auxiliary equipment, which is relatively cumbersome and cannot obtain the earphone feedback delay data in real time. Summary of the invention

[0004] In view of the above problems, the present application provides an ear return delay detection method, device, electronic device and storage medium, which can use the amount of cached data in the ear return system to calculate the cache delay caused by the cache in the ear return system, and combine the hardware delay of the ear return system to obtain the ear return delay of the ear return system. It does not require the assistance of external equipment, simplifies the ear return delay detection process, and facilitates the real-time acquisition of ear return delay data.

[0005] The embodiment of the present application is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present application provides an ear return delay detection method, which is applied to detecting the ear return delay of an ear return system, wherein the ear return system includes at least an audio collector and an audio player, and the method includes: determining a first cache delay according to the amount of collected cache data in the audio collector; determining the delay of the audio collector according to the first cache delay and the hardware delay of the audio collector; determining a second cache delay according to the amount of playback cache data in the audio player; determining the delay of the audio player according to the second cache delay and the hardware delay of the audio player; determining the ear return delay of the ear return system according to the delay of the audio collector and the delay of the audio player.

[0007] In a second aspect, the present application provides an ear return delay detection device, which is used to detect the ear return delay of an ear return system, wherein the ear return system at least includes an audio collector and an audio player, and the device includes: a first calculation module, used to determine a first cache delay according to the amount of collection cache data in the audio collector; an acquisition delay module, used to determine the delay of the audio collector according to the first cache delay and the hardware delay of the audio collector; a second calculation module, used to determine the second cache delay according to the amount of playback cache data in the audio player; a playback delay module, used to determine the delay of the audio player according to the second cache delay and the hardware delay of the audio player; a delay confirmation module, used to determine the ear return delay of the ear return system according to the delay of the audio collector and the delay of the audio player.

[0008] In some embodiments, the ear return delay detection device further includes a third calculation module, which is used to determine a third cache delay according to the amount of ear return cache data in the ear return cache area; the delay confirmation module is also used to add the delay of the audio collector, the delay of the audio player and the third cache delay to obtain the ear return delay of the ear return system.

[0009] In some embodiments, the ear return delay detection device also includes an overload delay module, which is used to determine the acquisition overload delay based on the amount of acquisition overload data of the audio collector; the acquisition delay module is also used to add the first cache delay, the hardware delay of the audio collector and the acquisition overload delay to obtain the delay of the audio collector.

[0010] Furthermore, the overload delay module includes an overload data calculation unit and an overload delay calculation unit; the overload data calculation unit is used to determine the collection overload data volume of the audio collector based on the number of collection overloads and the single collection data volume of the audio collector; the overload delay calculation unit is used to determine the collection overload delay based on the collection overload data volume of the audio collector and the audio collection parameters of the audio collector.

[0011] In some embodiments, the ear return delay detection device also includes an underload delay module; the underload delay module is used to determine the playback underload delay based on the amount of playback underload data of the audio player; the playback delay module is used to add the second cache delay, the hardware delay of the audio player and the playback underload delay to obtain the delay of the audio player.

[0012] Furthermore, the underload delay module includes an underload data calculation unit and an underload delay calculation unit; the underload data calculation unit is used to determine the amount of playback underload data of the audio player based on the number of playback underloads and the amount of single playback data of the audio player; the underload delay calculation unit is used to determine the playback underload delay based on the amount of playback underload data of the audio player and the audio playback parameters of the audio player.

[0013] In some implementations, the first calculation module is further configured to determine a first cache delay according to an audio collection parameter of the audio collector and an amount of collection cache data in the audio collector.

[0014] In some implementations, the second calculation module is further used to determine the second cache delay based on the audio playback parameters of the audio player and the amount of playback cache data in the audio player.

[0015] In a third aspect, the present application provides an electronic device, one or more processors; a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above method.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program code, and the program code can be called by a processor to execute the above method.

[0017] In a fifth aspect, the present application provides a computer program product, comprising a computer program / instructions, characterized in that the computer program / instructions implement the steps of the above method when executed by a processor.

[0018] The ear return delay detection method, device, electronic device and storage medium provided in the embodiments of the present application are applied to detecting the ear return delay of the ear return system, which includes at least an audio collector and an audio player. According to the amount of collected cache data in the audio collector, a first cache delay is determined; according to the first cache delay and the hardware delay of the audio collector, the delay of the audio collector is determined; according to the amount of playback cache data in the audio player, a second cache delay is determined; according to the second cache delay and the hardware delay of the audio player, the delay of the audio player is determined; according to the delay of the audio collector and the delay of the audio player, the ear return delay of the ear return system is determined; by adopting the method provided by the present application, the cache delay caused by the cache of the ear return system is calculated using the amount of cache data in the ear return system, and the ear return delay of the ear return system is obtained in combination with the hardware delay of the audio collector and the audio player in the ear return system, so that the detection of the ear return delay does not need to connect to an external device, simplifies the detection process of the ear return delay, and facilitates the real-time detection of the ear return delay.

[0019] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a schematic diagram of an implementation scenario involved in this application.

[0022] Figure 2 It is a flow chart of the ear-return delay detection method provided in this application.

[0023] Figure 3 This is another flow chart of the ear-return delay detection method provided in this application.

[0024] Figure 4 The embodiment of this application provides Figure 3 Schematic diagram of the process of step S320.

[0025] Figure 5 It is a schematic block diagram of the delay composition of the audio collector provided in an embodiment of the present application.

[0026] Figure 6 This is another flow chart of the earphone return delay detection method provided in this application.

[0027] Figure 7 The embodiment of this application provides Figure 6 Schematic diagram of the process of step S440.

[0028] Figure 8 It is a schematic block diagram of the delay composition of the audio player provided in an embodiment of the present application.

[0029] Fig. 9 It is a structural block diagram of the software ear monitoring system provided in an embodiment of the present application.

[0030] Fig.10 It is a schematic block diagram of an ear-return delay detection device provided in an embodiment of the present application.

[0031] Fig.11 It is a structural diagram of a computer system of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0033] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0034] To facilitate the understanding of this solution, several terms are explained below:

[0035] In-ear monitoring: refers to collecting the sound of the device and then playing it back on the device. It is generally used by the host to listen to the accompaniment, correct his or her own pronunciation, etc.

[0036] Earphone feedback delay: the time difference between the audio data being collected by the audio collector and being played by the audio player.

[0037] Hardware ear return: an ear return system that reads the collected audio data from the buffer area of ​​the audio collector and transmits it directly to the audio player for playback, without the need for an ear return application to process the audio data.

[0038] Software ear return: an ear return system that reads the collected audio data from the buffer area of ​​the audio collector and transmits it to the ear return application for audio processing, and then transmits the audio data processed by the ear return application to the audio player for playback.

[0039] Overrun: The speed of reading collected data cannot keep up with the speed of generating collected data, that is, data overload.

[0040] Underrun: The transmission speed of playback data cannot keep up with the consumption speed of playback data, that is, data underrun.

[0041] Sampling rate: refers to the number of audio samples recorded per second, which is measured in samples per second or hertz.

[0042] Channel: refers to the independent audio signals collected or played back at different spatial locations when recording or playing sound, so the number of channels is the number of sound sources when recording sound or the number of corresponding audio players (such as speakers) when playing back.

[0043] Cache: Cache is a buffer zone for data exchange (called Cache), which is a temporary place to store data (frequently used data). The essence of cache is to exchange space for time, sacrificing the real-time nature of data, and temporarily replacing the latest read data with stored data, thereby reducing data pressure and improving server performance.

[0044] With the rapid development of the media era, emerging industries with audio and video as the main working scenarios have emerged, such as online live broadcast and short video creation. In such audio and video scenarios, in-ear monitoring is an indispensable function. For example, for a chorus host, the host can correct his or her pronunciation in time through in-ear monitoring. Among them, the in-ear monitoring delay is an important indicator for evaluating in-ear monitoring, which directly affects the use effect of in-ear monitoring.

[0045] At present, the method of measuring earphone return delay is mainly to provide a regular audio waveform to the earphone return system, use an oscilloscope to view the collected audio waveform and the played audio waveform, calculate the waveform distance between the collected audio waveform and the played audio waveform, and thus calculate the earphone return delay; for example, connect a PC with a two-headed cable, input the played audio waveform into an oscilloscope, use the oscilloscope to measure the waveform distance between the played audio waveform and the input audio waveform, and thus obtain the earphone return data, but the operating steps of this type of measurement method are relatively complicated, and the quality data of the earphone return delay cannot be obtained at any time.

[0046] In order to solve the above problems, the present invention proposes an ear return delay detection method, which is applied to detect the ear return delay of an ear return system, wherein the ear return system at least includes an audio collector and an audio player, and determines a first cache delay according to the amount of collected cache data in the audio collector; determines the delay of the audio collector according to the first cache delay and the hardware delay of the audio collector; determines a second cache delay according to the amount of playback cache data in the audio player; determines the delay of the audio player according to the second cache delay and the hardware delay of the audio player; finally, determines the ear return delay of the ear return system according to the delay of the audio collector and the delay of the audio player; through the ear return delay detection method provided by the present application, the cache delay is calculated using the cache data amount in the ear return system, and the ear return delay is obtained in combination with the hardware delay of the ear return system, thereby avoiding the need to connect an external device to collect waveforms for delay detection, simplifying the ear return delay detection process, and greatly facilitating real-time detection of online ear return delay.

[0047] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0048] See also Figure 1 , Figure 1It is a schematic diagram of an implementation scenario involved in the present application, which is applied to an ear return system. The ear return system includes an audio collector 10, an audio player 20 and a processor 30. Data is transmitted between the audio collector 10, the audio player 20 and the processor 30 via wired or wireless means.

[0049] During the process of the audio collector 10 collecting audio and transmitting it to the audio player 20 for playback, the processor 30 can execute the method of the present application, calculate and output the delay of the ear return system from collecting audio to playing audio, that is, the ear return delay of the ear return system.

[0050] Exemplarily, in the process of the audio collector 10 collecting audio and transmitting it to the audio player 20 for playback, the processor 30 determines the first cache delay based on the amount of collected cache data in the audio collector 10; determines the delay of the audio collector 10 based on the first cache delay and the hardware delay of the audio collector 10; determines the second cache delay based on the amount of playback cache data in the audio player 20; determines the delay of the audio player 20 based on the second cache delay and the hardware delay of the audio player 20; determines the ear return delay of the ear return system based on the delay of the audio collector 10 and the delay of the audio player 20.

[0051] The audio collector 10 includes but is not limited to wired headphones, Bluetooth headphones, microphones, mobile phones and other electronic devices that can realize audio collection, and is not limited here.

[0052] The audio player 20 includes but is not limited to electronic devices that can play audio, such as mobile phones, speakers, and computers, and is not limited here.

[0053] It is worth mentioning that the audio collector 10, the audio player 20 and the processor 30 in the ear return system can be integrated in the same electronic device, or in two electronic devices (for example, the audio collector 10 is a separate electronic device, and the audio player 20 and the processor 30 are integrated in the same electronic device), or the audio collector 10, the audio player 20 and the processor 30 are respectively in different electronic devices, which are not specifically limited here. The processor 30 may include one or more processing cores. The processor 30 uses various interfaces and lines to connect the various parts of the entire ear return system; optionally, the processor 320 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 320 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem.

[0054] See also Figure 2 , Figure 2 This is a flow chart of the ear-return delay detection method provided by this application, which is applied to Figure 1 The in-ear monitoring system in Figure 1 The processor 30 in the ear return system shown in the figure specifically executes the method, which includes at least steps S210 to S250, which are described in detail as follows:

[0055] S210: Determine a first buffer delay according to the amount of collected buffer data in the audio collector.

[0056] The audio collector collects the audio analog signal of the sound source through the sound card, and then converts the audio analog signal into a digital signal and stores it in the buffer area to facilitate the subsequent reading of data. In this application, for the sake of distinction, the buffer area arranged for the audio collector is called the first buffer area.

[0057] In some embodiments, the size of the first buffer area of ​​the audio collector can be allocated after the audio collector starts collecting. After the audio collector starts collecting, the size of the first buffer area will not change dynamically; of course, in other embodiments, the size of the first buffer area can also be pre-set in the audio collector; further, when allocating the buffer area of ​​the audio collector, it can be a cache of a preset size, or it can be allocated according to the collection parameters of the audio collector; illustratively, the faster the collection speed of the audio collector, the larger the allocated first buffer area.

[0058] The amount of data in the acquisition buffer in the audio collector is the amount of data that has been collected and is to be read in the buffer area of ​​the audio collector. In some embodiments, the amount of data stored in the first buffer area can be read to obtain the amount of acquisition buffer data. It should be noted that due to the existence of the first buffer area, after the audio collector starts the acquisition, the collected audio data needs to be stored in the first buffer area first, and then the audio data is read from the first buffer area for subsequent processing (for example, for playback). For the ear return system, when the amount of data temporarily stored in the first buffer area reaches the maximum amount of data that can be stored in the first buffer area, the delay caused by the first buffer area is the largest. In this case, the maximum amount of data that can be stored in the first buffer area can be used as the acquisition buffer data amount. In some embodiments, the memory capacity of the first buffer area can be used as the maximum amount of data that can be stored in the first buffer area.

[0059] It can be understood that after the collected audio data is stored in the first buffer area, the audio data needs to wait in the first buffer area to be read. Thus, the process of waiting to be read causes the audio data to be read delayedly, or to be transmitted to the audio player delayedly. Therefore, the first cache delay can be understood as the delay caused by the audio data being temporarily stored in the first buffer area waiting to be read. The first cache delay can be understood as the waiting time for the audio data in the first buffer area to be read.

[0060] In some implementations, step S210 specifically includes:

[0061] S211. Determine a first buffer delay according to an audio collection parameter of the audio collector and a collection buffer data amount in the audio collector.

[0062] The audio acquisition parameters of the audio collector include but are not limited to the sampling rate, sampling accuracy, number of channels, etc. of the audio collector.

[0063] It should be noted that the audio collection parameters of the audio collector need to be pre-configured before the audio collector starts collecting, and after the audio collector starts collecting, the audio collection parameters remain stable.

[0064] In some embodiments, the audio acquisition parameters of the audio collector include the sampling rate and the number of channels of the audio collector. Based on the audio acquisition parameters of the audio collector, the data reading speed in the first buffer area can be determined, that is, the amount of data read per unit time, and then, the corresponding time for the audio data in the first buffer area to wait for reading, that is, the first buffer delay, can be determined according to the amount of acquisition buffer data and the data reading speed in the first buffer area.

[0065] Specifically, the first buffer delay is determined according to the sampling rate and the number of channels of the audio collector and the amount of collected buffer data in the audio collector, which can be calculated by the following formula:

[0066] That is: the first cache delay = the amount of data in the acquisition cache / (sampling rate * number of channels * number of bytes per sampling point), where the sampling rate and the number of channels are the sampling rate and the number of channels of the audio collector.

[0067] The number of bytes at each sampling point is the amount of data for each audio sample. It should be noted that the number of bytes at each sampling point is pre-configured in the system configuration of the audio collector, and the number of bytes at each sampling point can be determined by reading the system configuration of the audio collector.

[0068] It can be understood that since the sampling rate represents the number of audio samples of a channel extracted by the audio collector within 1 second, the number of bytes at each sampling point represents the amount of data occupied by each audio sample, and the number of channels represents the number of sound sources collected simultaneously by the audio collector, the amount of audio data collected by the audio collector within 1 second can be calculated by sampling rate * number of channels * number of bytes at each sampling point, that is, the amount of data read per unit time.

[0069] Furthermore, considering that the delay of the audio collector is in the millisecond level, in some embodiments, the above calculation formula can be expressed as:

[0070] First cache delay (ms) = 1000*acquisition cache data volume / (sampling rate*number of channels*number of bytes per sampling point), where the sampling rate and number of channels are the sampling rate and number of channels of the audio collector.

[0071] Through the ear return delay detection method provided in the embodiment of the present application, when the audio collector collects audio, by obtaining the amount of collection cache data of the audio collector, and obtaining the sampling rate and number of channels of the audio collector, the first cache delay of the audio collector caused by the setting of the cache area can be calculated.

[0072] S220: Determine the delay of the audio collector according to the first buffer delay and the hardware delay of the audio collector.

[0073] The hardware delay of the audio collector refers to the time difference from when the audio data is collected by the audio collector from the sound source to when the audio data is converted into a signal and then transmitted to the buffer area of ​​the audio collector.

[0074] The hardware delay of the audio collector is the delay caused by the transmission line, signal conversion, etc. of the audio collector due to the hardware of the audio collector. Therefore, for a specific audio collector, its hardware delay is a fixed value.

[0075] In actual application, the hardware delay of the audio collector will be configured in the system configuration in advance to optimize the ear return system according to the hardware delay of the audio collector; therefore, in an embodiment of the present application, the hardware delay of the audio collector can be obtained by reading the system configuration of the audio collector.

[0076] It is understandable that, since the hardware delay of the audio collector is a fixed value, when multiple delay detections are performed on the same audio collector, it is only necessary to read the hardware delay of the audio collector during the first detection.

[0077] In some embodiments, the first buffer delay and the hardware delay of the audio collector may be added to obtain the delay of the audio collector.

[0078] S230, determining a second buffer delay according to the amount of playback buffer data in the audio player. The audio player is used to play audio signals (or audio data). Usually, a buffer area is provided in the audio player, and the buffer area is used to temporarily store the audio data to be played. In the present application, the buffer area provided in the audio player for temporarily storing the audio data to be played is referred to as a second buffer area. The audio data is temporarily stored in the second buffer area and waits to be played.

[0079] Among them, the size of the second buffer area of ​​the audio player can be allocated or pre-allocated after the audio player starts playing. Usually, after the audio player starts playing, the size of the second buffer area will not change dynamically; further, when allocating the second buffer area of ​​the audio player, a cache of a preset size can be allocated, or a cache can be allocated according to the playback parameters of the audio player; illustratively, the faster the playback speed of the audio player, the larger the allocated second buffer area.

[0080] The amount of playback cache data in the audio player is also the amount of data cached in the second cache area, that is, the amount of audio data that has been read to be played; in some embodiments, the amount of data stored in the second cache area can be read to obtain the amount of playback cache data. It should be noted that due to the existence of the second cache area, after the audio player starts playing, the audio data needs to be stored in the second cache area first, and then the audio data is read from the second cache area for playback. For the ear return system, when the amount of data temporarily stored in the second cache area reaches the maximum amount of data that can be stored in the second cache area, the delay caused by the second cache area is the largest. In this case, the maximum amount of data that can be stored in the second cache area can be used as the amount of playback cache data. In some embodiments, the memory capacity of the second cache area can be used as the maximum amount of data that can be stored in the second cache area.

[0081] It can be understood that after the audio data read from the audio collector is stored in the second buffer area, the audio data needs to wait in the second buffer area to be played. Thus, the process of waiting to be played causes the audio data to be read delayedly or played delayedly. Therefore, the second cache delay can be understood as the delay caused by the audio data being temporarily stored in the second buffer area waiting to be played. The second cache delay can be understood as the waiting time for the audio data in the second buffer area to be played.

[0082] In some implementations, step S230 specifically includes:

[0083] S231. Determine a second buffer delay according to an audio playback parameter of the audio player and an amount of playback buffer data in the audio player.

[0084] Among them, the audio playback parameters of the audio player include but are not limited to the sampling rate, sampling accuracy, number of channels, etc. of the audio player.

[0085] It should be noted that the audio playback parameters of the audio player need to be pre-configured before the audio player starts playing. Usually, after the audio player starts playing, the audio playback parameters remain stable.

[0086] In an embodiment of the present application, in order to make the audio output by the audio player meet specific output requirements, the audio data collected by the audio collector needs to be resampled. Therefore, in an embodiment of the present application, the audio playback parameters of the audio player include the sampling rate and number of channels of the audio player.

[0087] It should be noted that the sampling rate and number of channels of the audio player are completely independent parameters from the sampling rate and number of channels of the audio collector involved in the aforementioned embodiment, and the two may be the same or different.

[0088] The data reading speed in the second cache area, i.e., the amount of data read per unit time, can be determined based on the audio acquisition parameters of the audio player. Subsequently, the time the audio data in the second cache area waits to be read, i.e., the second cache delay, can be determined based on the amount of playback cache data and the data reading speed in the second cache area.

[0089] Specifically, in some embodiments, the second buffer delay is determined according to the sampling rate and the number of channels of the audio player and the amount of play buffer data in the audio player, and can be calculated by the following calculation formula:

[0090] That is: the second cache delay = the amount of play cache data / (sampling rate*number of channels*number of bytes per sampling point), where the sampling rate and number of channels are the sampling rate and number of channels of the audio player.

[0091] Among them, the number of bytes at each sampling point is also the data amount of each audio sample, or refers to the number of bytes of a single sampled audio data; it should be noted that the number of bytes at each sampling point is pre-configured in the system configuration of the audio player, and the number of bytes at each sampling point can be determined by reading the system configuration of the audio player.

[0092] Furthermore, considering that the delay of the audio collector is in the millisecond level, in the embodiment of the present application, the above calculation formula can be expressed as:

[0093] Second cache delay (ms) = 1000*playback cache data volume / (sampling rate*number of channels*number of bytes per sampling point), where the sampling rate and number of channels are the sampling rate and number of channels of the audio player.

[0094] Through the ear return delay detection method provided in the embodiment of the present application, when an audio player plays audio, by obtaining the amount of playback cache data of the audio player, and obtaining the sampling rate and number of channels of the audio player, the second cache delay of the audio player caused by the cache area setting can be calculated.

[0095] S240: Determine the delay of the audio player according to the second cache delay and the hardware delay of the audio player.

[0096] The hardware delay of the audio player refers to the delay in the process of converting the audio data into a signal after the audio data is read from the second buffer area of ​​the audio player and then playing it.

[0097] The hardware delay of an audio player is the delay caused by the audio player's playback circuit, signal conversion, and other factors caused by the audio player's hardware. Therefore, for a specific audio player, its hardware delay is a fixed value.

[0098] In actual application, the hardware delay of the audio player will be configured in the system configuration in advance to optimize the ear return system according to the hardware delay of the audio player; therefore, in an embodiment of the present application, the hardware delay of the audio player can be obtained by reading the system configuration of the audio player.

[0099] It is understandable that, since the hardware delay of the audio player is a fixed value, when multiple delay detections are performed on the same audio player, it is only necessary to read the hardware delay of the audio player during the first detection.

[0100] In some embodiments, the second cache delay and the hardware delay of the audio player may be added together to obtain the delay of the audio player.

[0101] S250: Determine the ear return delay of the ear return system according to the delay of the audio collector and the delay of the audio player.

[0102] In some embodiments, the delay of the frequency collector and the delay of the audio player are added together to obtain the ear return delay of the ear return system.

[0103] The ear return delay detection method provided in the embodiment of the present application obtains the sampling rate, number of channels and acquisition buffer data of the audio collector, calculates the first buffer delay of the audio collector, and obtains the delay of the audio collector according to the first buffer delay and the hardware delay of the audio collector; obtains the second buffer delay of the audio player by obtaining the sampling rate, number of channels and playback buffer data of the audio player, and obtains the delay of the audio player according to the second buffer delay and the hardware delay of the audio player, thereby obtaining the delay of the ear return system; through the above method, considering that the ear return delay of the ear return system includes the delay caused by the audio collector and the delay caused by the audio player, and the first buffer delay of the audio collector is calculated using the acquisition buffer data volume of the audio collector, and the second buffer delay of the audio player is calculated using the playback buffer data volume of the audio player, and then the ear return delay of the ear return system is obtained in combination with the hardware delay of the audio collector and the hardware delay of the audio player in the system parameters, so that the ear return delay of the ear return system can be determined without using additional detection equipment (such as an oscilloscope), simplifying the delay detection process of the ear return system, and facilitating real-time calculation of the ear return delay of the ear return system.

[0104] In other embodiments, considering that the data transmission process of the ear return system involves multiple data collection and reading, when the speed of collection and reading does not match, transmission errors are likely to occur, resulting in changes in the delay of the ear return system. Exemplary, common transmission errors include cache overload (overrun) and cache underrun (underrun). When the audio data in the first cache area of ​​the audio collector is not read in time, an overflow occurs. For example, when the collection speed of the audio collector is too fast, the audio data in the first cache area of ​​the audio collector will overflow, resulting in partial data loss, which is also called overrun; for another example, the audio data is not sent to the second cache area of ​​the audio player in time, resulting in a cache underrun in the second cache area.

[0105] In order to accurately detect the ear return delay of the ear return system when the ear return system overruns, the present application also provides another flow chart of the ear return delay detection method, such as Figure 3 , Figure 3 Another flow chart provided in an embodiment of the present application is given, and the ear-return delay detection method includes:

[0106] S310: Determine a first buffer delay according to the amount of collected buffer data in the audio collector.

[0107] Among them, the process of step S310 is the same as or similar to that of step S210 in the aforementioned embodiment. Therefore, the specific description of step S310 can refer to the specific description of step S210 in the aforementioned embodiment, which will not be repeated here.

[0108] S320: Determine a collection overload delay based on the collection overload data volume of the audio collector.

[0109] Among them, the amount of data collected by the audio collector is overloaded, that is, the amount of data lost by the audio collector due to cache overflow; further, the data included in the lost data may be data already stored in the cache, that is, old data, or may be the latest collected data that cannot be stored in the cache, and can be set according to actual needs.

[0110] It should be noted that, since the amount of data collected by the audio collector each time is a fixed value, the amount of data collected by the audio collector when it is overloaded can be calculated by the number of times the audio collector is overloaded.

[0111] In some embodiments, Figure 4 As shown, Figure 4 The embodiment of the present application provides Figure 3 Schematic diagram of the process of step S320, step S320 specifically includes:

[0112] S321. Determine the amount of data collected by the audio collector based on the number of collection overloads and the amount of data collected at a single time.

[0113] It should be noted that every time a cache overflow occurs in the audio collector, that is, the audio collector of the present application is overloaded, the counter in the ear return system will record an overrun; by reading the number of overruns currently recorded in the ear return system, the number of collection overloads of the audio collector can be obtained.

[0114] It should be noted that after the ear return system completes the ear return task, that is, after the ear return system stops collecting and playing, the counter in the ear return system will be reset to facilitate delay detection of the ear return system again.

[0115] Among them, the single acquisition data volume of the audio collector refers to the amount of data required by the audio collector for each processing, that is, the amount of data written to the buffer each time; exemplarily, the audio collector needs to process n frames of data each time, where each frame contains m audio samples, then the single acquisition data volume is n*m*the number of bytes at each sampling point.

[0116] It should be noted that after the audio collector starts collecting data, the single collection data volume of the audio collector will remain stable, and the single collection data volume can be obtained by reading the system setting parameters of the audio collector.

[0117] In some embodiments, if the single acquisition data volume of the audio collector is calculated based on a single channel, the number of channels needs to be considered when calculating the acquisition overload data volume for multiple channels, that is, the acquisition overload data volume of the audio collector can be calculated by the following calculation formula:

[0118] The amount of data collected overload = the number of collection overloads * the amount of data collected in a single time * the number of channels, where the number of channels refers to the number of channels of the audio collector.

[0119] S322: Determine a collection overload delay based on the collection overload data volume of the audio collector and the audio collection parameters of the audio collector.

[0120] The audio acquisition parameters of the audio collector include but are not limited to the sampling rate, sampling accuracy, number of channels, etc. of the audio collector.

[0121] It should be noted that the audio collection parameters of the audio collector need to be pre-configured before the audio collector starts collecting, and after the audio collector starts collecting, the audio collection parameters remain stable.

[0122] In the embodiment of the present application, the audio collection parameters of the audio collector include the sampling rate and the number of channels of the audio collector.

[0123] Considering that the acquisition overload delay of the audio collector is in the millisecond level, the acquisition overload delay of the audio collector can be calculated by the following formula:

[0124] Collection overload delay (ms) = 1000 * collection overload data volume / (sampling rate * number of channels * number of bytes per sampling point).

[0125] S330: Add the first cache delay, the hardware delay of the audio collector, and the collection overload delay to obtain the delay of the audio collector.

[0126] Among them, step S330 is the same or similar to the process of step S220 in the aforementioned embodiment. The only difference is that in step S330, it is necessary to consider the acquisition overload delay of the audio collector to obtain the delay of the audio collector. Therefore, for the specific description of step S330, please refer to the specific description of step S220 in the aforementioned embodiment, which will not be repeated here.

[0127] To understand the delay components of the audio collector, please refer to Figure 5 , Figure 5 The delay composition diagram of the audio collector is given. The delay of the audio collector includes:

[0128] Acquisition overload delay (also called overrun delay) refers to the delay caused by audio data overflow in the first buffer;

[0129] Cache data delay (i.e., first cache delay in this application) refers to the delay caused by the audio data being temporarily stored in the first cache area;

[0130] And the hardware delay of the audio collector, among which the acquisition hardware delay specifically includes the delay of the acquisition process, the delay of signal conversion and the delay of data transmission; specifically, the delay of the acquisition process refers to the time difference between the sound being emitted from the sound source and being collected by the sound card of the audio collector; the delay of signal conversion refers to the time difference between the audio collector converting the analog signal into a digital signal; the delay of data transmission refers to the time difference between the converted digital signal being transmitted to the buffer area for caching.

[0131] In summary, the delay of the audio collector = the hardware delay of the audio collector + the cache data delay + the collection overload delay.

[0132] S340: Determine a second buffer delay according to the amount of playback buffer data in the audio player.

[0133] S350. Determine the delay of the audio player according to the second cache delay and the hardware delay of the audio player.

[0134] S360: Determine the ear return delay of the ear return system according to the delay of the audio collector and the delay of the audio player.

[0135] Among them, steps S340-S360 are the same or similar to the processes of steps S230-S250 in the aforementioned embodiment. Therefore, the specific description of steps S340-S360 can refer to the specific description of steps S230-S250 in the aforementioned embodiment, which will not be repeated here.

[0136] The ear return delay detection method provided in the embodiment of the present application, based on respectively calculating the hardware delay and cache delay of the audio collector, as well as the hardware delay and cache delay of the audio player, further calculates the acquisition overload delay caused by cache overload of the audio collector, thereby accurately detecting the ear return delay of the ear return system when the audio acquisition is overloaded, thereby improving the accuracy of the ear return delay.

[0137] In order to accurately detect the ear return delay of the ear return system when underrun occurs, the present application also provides another flow chart of the ear return delay detection method, such as Figure 6 , Figure 6 Another flow chart provided in an embodiment of the present application is given, and the ear-return delay detection method includes:

[0138] S410: Determine a first buffer delay according to the amount of collected buffer data in the audio collector.

[0139] S420: Obtain a delay of the audio collector based on the first cache delay and the hardware delay of the audio collector.

[0140] S430: Determine a second buffer delay according to the amount of playback buffer data in the audio player.

[0141] Among them, steps S410-S430 are the same or similar to the processes of steps S210-S230 in the aforementioned embodiment. Therefore, the specific description of steps S410-S430 can refer to the specific description of steps S210-S230 in the aforementioned embodiment, which will not be repeated here.

[0142] S440: Determine a playback underload delay based on the amount of playback underload data of the audio player.

[0143] The amount of under-play data of the audio player is the amount of data missing during the playback of the audio collector.

[0144] It should be noted that since the amount of data required for each playback of the audio player is pre-set, the amount of playback underload data is related to the number of times the audio player experiences playback underload, and the amount of playback underload data of the audio player can be calculated by the number of times the audio player experiences playback underload.

[0145] In some embodiments, Figure 7As shown, Figure 7 The embodiment of the present application provides Figure 6 Schematic diagram of the process of step S440, step S440 specifically includes:

[0146] S441. Determine the amount of under-play data of the audio player based on the number of under-play times of the audio player and the amount of data played at a single time.

[0147] It should be noted that every time a cache underrun occurs in the audio player, that is, a playback underrun occurs in the audio collector of the present application, a counter in the ear return system will record an underrun; by reading the number of underruns currently recorded in the ear return system, the number of playback underruns of the audio player can be obtained.

[0148] The single playback data volume of the audio player refers to the amount of data required by the audio player for each playback, that is, the amount of data that needs to be read from the playback buffer area of ​​the audio player for each playback.

[0149] It should be noted that after the audio player starts playing, the single playback data volume of the audio player will remain stable, and the single playback data volume can be obtained by reading the system setting parameters of the audio player.

[0150] In some embodiments, if the single playback data volume of the audio player is calculated based on a single channel, the number of channels needs to be considered when calculating the playback underload data volume for multiple channels, that is, the playback underload data volume of the audio player can be calculated by the following calculation formula:

[0151] The amount of playback underload data = the number of playback underloads * the amount of single playback data * the number of channels, where the number of channels refers to the number of channels of the audio player.

[0152] S442: Determine a playback underload delay based on the amount of playback underload data of the audio player and audio playback parameters of the audio player.

[0153] Among them, the audio playback parameters of the audio player include but are not limited to the sampling rate, sampling accuracy, number of channels, etc. of the audio player.

[0154] It should be noted that the audio playback parameters of the audio player need to be pre-configured before the audio player starts playing, and after the audio player starts playing, the audio playback parameters remain stable.

[0155] In an embodiment of the present application, the audio playback parameters of the audio player include the sampling rate and number of channels of the audio player.

[0156] Considering that the playback underload delay of the audio player is in milliseconds, the playback underload delay of the audio player can be calculated by the following formula:

[0157] Playback underload delay (ms) = 1000*playback underload data volume / (sampling rate*number of channels*number of bytes per sampling point), where the sampling rate and number of channels are the sampling rate and number of channels of the audio player.

[0158] S450: Obtain the delay of the audio player by adding the second cache delay, the hardware delay of the audio player and the playback underload delay.

[0159] Among them, step S450 is the same or similar to the process of step S240 in the aforementioned embodiment. The only difference is that in step S450, it is necessary to consider the playback underload delay of the audio player to obtain the delay of the audio player. Therefore, for the specific description of step S450, please refer to the specific description of step S240 in the aforementioned embodiment, which will not be repeated here.

[0160] To understand the delay components of an audio player, see Figure 8 , Figure 8 The delay composition diagram of the audio player is given. The delay of the audio player includes:

[0161] Playback underload delay (also called underrun delay) refers to the delay caused by the audio data not being delivered to the second buffer in time;

[0162] Cache data delay (i.e., second cache delay in this application) refers to the delay caused by the audio data being temporarily stored in the second cache area;

[0163] And the hardware delay of the audio player; wherein, the hardware delay of the audio player specifically includes the delay of data transmission, the delay of signal conversion and the delay of the playback process; specifically, the delay of data transmission refers to the time difference generated by reading the data in the second buffer area and transmitting it to the speaker; the delay of signal conversion refers to the time difference when the audio player converts the digital signal into an analog signal; the delay of the playback process refers to the time difference generated by transmitting the converted analog signal to the speaker for playback.

[0164] In summary, the delay of the audio player = the hardware delay of the audio player + the cache data delay + the playback underload delay.

[0165] S460: Determine the ear return delay of the ear return system according to the delay of the audio collector and the delay of the audio player.

[0166] Among them, the process of step S460 is the same as or similar to that of step S250 in the aforementioned embodiment. Therefore, the specific description of step S460 can refer to the specific description of step S250 in the aforementioned embodiment, which will not be repeated here.

[0167] The ear return delay detection method provided in the embodiment of the present application, based on respectively calculating the hardware delay and cache delay of the audio collector, and the hardware delay and cache delay of the audio player, further calculates the playback underload delay of the audio player caused by underload of the second buffer area (also called underrun), thereby accurately detecting the ear return delay of the ear return system when the audio playback is underloaded, thereby improving the accuracy of the ear return delay.

[0168] It can be understood that if the ear return system has a first cache overload and a second cache underload at the same time, the ear return delay of the ear return system includes the delay of the audio collector and the delay of the audio player. The delay of the audio collector needs to include the collection overload delay of the audio collector, and the delay of the audio player needs to include the playback underload delay of the audio player.

[0169] It should be noted that in the ear return delay detection method provided in the above embodiment, if the ear return function is realized only by hardware, that is, the ear return system only includes an audio collector and an audio player, in this case, after the audio data in the first buffer area of ​​the audio collector is read, it is directly transmitted to the second buffer area of ​​the audio player for temporary storage, and no intermediate processing process is involved, that is, the ear return system is a hardware ear return. In this case, the ear return delay of the ear return system is equal to the sum of the delay of the audio collector and the delay of the audio player. In other embodiments, the audio data read from the first buffer area is processed by the ear return application running in the ear return system, such as setting the volume, sound effects, etc., thereby forming a software ear return. In this case, a cache area for caching the audio data to be processed is set for the ear return application on the device running the ear return application in the ear return system, and the cache area is called the ear return cache area.

[0170] For easier understanding, see Fig. 9 , Fig. 9 A structural block diagram of a software ear return system is given. The software ear return system includes an audio collector, an ear return application and an audio player, wherein the ear return application is provided with an ear return buffer area.

[0171] The audio collector is responsible for starting the collection, then reading the collected data and transmitting it to the ear return application;

[0172] The ear return application is responsible for processing the data such as sound effects and volume according to the user's needs, and then storing it in the ear return buffer area;

[0173] The audio player is responsible for starting playback, then reading data from the earphone return buffer and transmitting it to the system for playback.

[0174] In order to facilitate the detection of the delay of the software ear return, the ear return delay detection method provided in the present application also includes: determining the third cache delay according to the amount of ear return cache data in the ear return cache area. Correspondingly, in this embodiment, step S250 includes: adding the delay of the audio collector, the delay of the audio player and the third cache delay to obtain the ear return delay of the ear return system.

[0175] Among them, the amount of ear return buffer data is the amount of audio data temporarily stored in the ear return buffer area waiting to be processed by the ear return application.

[0176] In some embodiments, the size of the ear return buffer area can be allocated after the ear return system is started. After the ear return system is started, the size of the ear return buffer area will not change dynamically; of course, in other embodiments, the size of the ear return buffer area can also be pre-set in the ear return system; further, when allocating the ear return buffer area, a cache of a preset size can be allocated, or the cache can be allocated according to the parameters of the audio collector and the audio player.

[0177] The amount of ear return cache data in the ear return cache area is the amount of data stored in the ear return cache area, which is read from the first cache area and is waiting to be processed and sent to the second cache area. In some embodiments, the amount of data stored in the ear return cache area can be read to obtain the amount of ear return cache data. It should be noted that due to the existence of the ear return cache area, after the ear return system is started, the audio data in the first cache area needs to be read into the ear return cache area for temporary storage, and then transmitted to the second cache area after being processed by the ear return application. For the ear return system, when the amount of data temporarily stored in the ear return cache area reaches the maximum amount of data that can be stored in the ear return cache area, the delay caused by the ear return cache area is the largest. In this case, the maximum amount of data that can be stored in the ear return cache area can be used as the amount of ear return cache data. In some embodiments, the memory capacity of the ear return cache area can be used as the maximum amount of data that can be stored in the ear return cache area.

[0178] It can be understood that after the audio data in the first buffer area is stored in the ear return buffer area, the audio data needs to wait in the ear return buffer area to be processed by the ear return application. Thus, the process of waiting to be processed causes the audio data to be delayed in transmitting to the second buffer area. Therefore, the third cache delay can be understood as the delay caused by the audio data being temporarily stored in the ear return buffer area waiting to be processed by the ear return application. The third cache delay can be understood as the waiting time for the audio data in the ear return buffer area to be processed.

[0179] In some embodiments, the third buffer delay may be calculated by the following formula:

[0180] The third cache delay (ms) = 1000 * ear-return cache data volume (bytes) / (sampling rate * number of channels * number of bytes per sampling point).

[0181] It should be noted that since no resampling is required between the buffer area of ​​the audio player and the ear return buffer area, in the above calculation formula, the sampling rate and the number of channels are the sampling rate and the number of channels of the audio collector.

[0182] The ear return delay detection method provided in the embodiment of the present application can be applied to the ear return delay detection of the software ear return. On the basis of the ear return delay of the hardware ear return, the ear return delay of the software ear return is obtained by calculating the third cache delay of the ear return cache area, thereby enriching the applicable scenarios of the ear return delay detection method of the present application.

[0183] It should be noted that, in some other embodiments, before executing the ear return delay detection method of the present application, the type of the ear return system can also be obtained, wherein the type of the ear return system includes at least hardware ear return and software ear return.

[0184] Furthermore, the type of the ear return system can be obtained by manual confirmation by the user; or by obtaining system configuration parameters and determining whether there is an ear return cache area according to the system configuration parameters; if there is an ear return cache area, it is determined to be a software ear return; if not, it is determined to be a hardware ear return.

[0185] See also Fig.10 The present application also provides an ear return delay detection device, which is used to detect the ear return delay of an ear return system. The ear return system at least includes an audio collector and an audio player. The ear return delay detection device 500 includes:

[0186] A first calculation module 510, configured to determine a first buffer delay according to the amount of collected buffer data in the audio collector;

[0187] A collection delay module 520, used to determine the delay of the audio collector according to the first buffer delay and the hardware delay of the audio collector;

[0188] A second calculation module 530, configured to determine a second buffer delay according to the amount of playback buffer data in the audio player;

[0189] A play delay module 540, configured to determine a delay of the audio player according to the second buffer delay and a hardware delay of the audio player;

[0190] The delay confirmation module 550 is used to determine the ear return delay of the ear return system according to the delay of the audio collector and the delay of the audio player.

[0191] In some embodiments, the ear return delay detection device 500 further includes a third calculation module, which is used to determine a third cache delay according to the amount of ear return cache data in the ear return cache area; the delay confirmation module 550 is also used to add the delay of the audio collector, the delay of the audio player and the third cache delay to obtain the ear return delay of the ear return system.

[0192] In some embodiments, the ear return delay detection device 500 also includes an overload delay module, which is used to determine the acquisition overload delay based on the amount of acquisition overload data of the audio collector; the acquisition delay module 520 is also used to add the first cache delay, the hardware delay of the audio collector and the acquisition overload delay to obtain the delay of the audio collector.

[0193] Furthermore, the overload delay module includes an overload data calculation unit and an overload delay calculation unit; the overload data calculation unit is used to determine the collection overload data volume of the audio collector based on the number of collection overloads and the single collection data volume of the audio collector; the overload delay calculation unit is used to determine the collection overload delay based on the collection overload data volume of the audio collector and the audio collection parameters of the audio collector.

[0194] In some embodiments, the ear return delay detection device 500 also includes an underload delay module; the underload delay module is used to determine the playback underload delay based on the amount of playback underload data of the audio player; the playback delay module 540 is also used to add the second cache delay, the hardware delay of the audio player and the playback underload delay to obtain the delay of the audio player.

[0195] Furthermore, the underload delay module includes an underload data calculation unit and an underload delay calculation unit; the underload data calculation unit is used to determine the amount of playback underload data of the audio player based on the number of playback underloads and the amount of single playback data of the audio player; the underload delay calculation unit is used to determine the playback underload delay based on the amount of playback underload data of the audio player and the audio playback parameters of the audio player.

[0196] In some implementations, the first calculation module 510 is further configured to determine the first buffer delay according to the audio collection parameters of the audio collector and the amount of collection buffer data in the audio collector.

[0197] In some implementations, the second calculation module 520 is further configured to determine a second cache delay based on an audio playback parameter of the audio player and an amount of playback cache data in the audio player.

[0198] See also Fig.11Based on the earphone return delay detection method provided in the above embodiment, the embodiment of the present application also provides another electronic device including a processor that can execute the above method. The electronic device can be a terminal device, and the terminal device can be a smart phone, a tablet computer, a computer or a portable computer. The present application also provides an electronic device, the electronic device including:

[0199] Fig.11 The structure diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. The electronic device may be a terminal as described above, and is used to implement the ear return delay detection method provided in the present application. It should be noted that: Fig.11 The computer system 1400 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application. In other embodiments, the electronic device may also include an audio player and an audio collector. In this case, the electronic device may be used as an ear return system, and the electronic device may determine its own ear return delay according to the method of the present application.

[0200] like Fig.11 As shown, the computer system 1400 includes a central processing unit (CPU) 1401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1402 or the program loaded from the storage part 1408 to the random access memory (RAM) 1403, such as executing the method in the above embodiment. In RAM 1403, various programs and data required for system operation are also stored. CPU 1401, ROM 1402 and RAM 1403 are connected to each other through bus 1404. Input / output (I / O) interface 1405 is also connected to bus 1404.

[0201] The following components are connected to the I / O interface 1405: an input section 1406 including a keyboard, a mouse, a microphone, etc.; an output section 1407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1408 including a hard disk, etc.; and a communication section 1409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1409 performs communication processing via a network such as the Internet. A drive 1410 is also connected to the I / O interface 1405 as needed. A removable medium 1411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1410 as needed so that a computer program read therefrom is installed into the storage section 1408 as needed.

[0202] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1409, and / or installed from a removable medium 1411. When the computer program is executed by a central processing unit (CPU) 1401, various functions defined in the system of the present application are executed.

[0203] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0204] In summary, the ear return delay detection method, device, electronic device and storage medium provided by the present application, when the ear return system is a hardware ear return, determines the first cache delay according to the amount of collection cache data in the audio collector; determines the collection overload delay according to the amount of collection overload data in the audio collector; determines the delay of the audio collector according to the first cache delay, the hardware delay of the audio collector and the collection overload delay; at the same time, determines the second cache delay according to the amount of playback cache data in the audio player; determines the playback underload delay according to the amount of playback underload data in the audio player; determines the delay of the audio player according to the second cache delay, the playback underload delay and the hardware delay of the audio player; finally determines the ear return delay of the ear return system according to the delay of the audio collector and the delay of the audio player; When the return system is a software ear return, on the basis of the ear return delay of the hardware ear return, the third cache delay is determined according to the amount of ear return cache data in the ear return buffer area; the delay of the audio collector, the delay of the audio player and the third cache delay are added to obtain the ear return delay of the ear return system; through the ear return delay detection method provided by the present application, various system information in the ear return system (such as acquisition parameters, playback parameters, transmission error information, cache data volume, hardware delay, etc.) are used to calculate the delay of each part of the ear return system, so as to obtain the ear return delay of the ear return system, so that there is no need to perform delay detection by connecting external devices, which simplifies the ear return delay detection process, greatly facilitates the real-time detection of online ear return delay, and is applicable to both hardware ear return and software ear return, enriching the applicable scenarios of ear return delay detection.

[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting earphone delay, characterized in that: The method is applied to detecting the ear return delay of an ear return system, wherein the ear return system includes at least an audio collector and an audio player, and the method includes: Determining a first buffer delay according to the amount of collected buffer data in the audio collector; Determining a delay of the audio collector according to the first buffer delay and a hardware delay of the audio collector; Determining a second buffer delay according to the amount of playback buffer data in the audio player; Determining a delay of the audio player according to the second buffer delay and a hardware delay of the audio player; The ear return delay of the ear return system is determined according to the delay of the audio collector and the delay of the audio player.

2. The method according to claim 1, characterized in that The ear return system is also provided with an ear return buffer area, and the method comprises: Determining a third buffer delay according to the amount of ear return buffer data in the ear return buffer area; The step of determining the ear return delay of the ear return system according to the delay of the audio collector and the delay of the audio player comprises: The delay of the audio collector, the delay of the audio player and the third cache delay are added together to obtain the ear return delay of the ear return system.

3. The method according to claim 1, characterized in that The method further comprises: Determining a collection overload delay based on the amount of collection overload data of the audio collector; The step of determining the delay of the audio collector according to the first buffer delay and the hardware delay of the audio collector comprises: The first buffer delay, the hardware delay of the audio collector and the acquisition overload delay are added together to obtain the delay of the audio collector.

4. The method according to claim 3, characterized in that The determining of the acquisition overload delay based on the acquisition overload data amount of the audio collector includes: Determining the amount of collection overload data of the audio collector based on the number of collection overloads and the amount of single collection data of the audio collector; Based on the amount of collection overload data of the audio collector and the audio collection parameters of the audio collector, a collection overload delay is determined.

5. The method according to claim 1, characterized in that The method further comprises: Determining a playback underload delay based on the amount of playback underload data of the audio player; The step of determining the delay of the audio player according to the second buffer delay and the hardware delay of the audio player comprises: The second cache delay, the hardware delay of the audio player and the playback underload delay are added together to obtain the delay of the audio player.

6. The method according to claim 5, characterized in that The determining of the playback underload delay based on the amount of playback underload data of the audio player includes: Determining the amount of under-play data of the audio player based on the number of under-play times of the audio player and the amount of data played once; The playback underload delay is determined based on the amount of playback underload data of the audio player and the audio playback parameters of the audio player.

7. The method according to claim 1, characterized in that The step of determining the first buffer delay according to the amount of collected buffer data in the audio collector includes: A first buffer delay is determined according to the audio collection parameters of the audio collector and the amount of collection buffer data in the audio collector.

8. The method according to claim 1, characterized in that The determining the second buffer delay according to the amount of playback buffer data in the audio player includes: The second cache delay is determined according to the audio playback parameters of the audio player and the amount of playback cache data in the audio player.

9. An ear-return delay detection device, characterized in that: The invention is applied to detecting the ear return delay of an ear return system, wherein the ear return system comprises at least an audio collector and an audio player, including: A first calculation module, used for determining a first cache delay according to the amount of collection cache data in the audio collector; A collection delay module, used for determining the delay of the audio collector according to the first buffer delay and the hardware delay of the audio collector; A second calculation module, used to determine a second buffer delay according to the amount of playback buffer data in the audio player; A play delay module, used to determine the delay of the audio player according to the second cache delay and the hardware delay of the audio player; The delay confirmation module is used to determine the ear return delay of the ear return system according to the delay of the audio collector and the delay of the audio player.

10. An electronic device, characterized in that: include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores program codes, and the program codes can be called by a processor to execute the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

Images