A method, device, equipment and storage medium for delay estimation

Abstract

The embodiments of the present application disclose a method, device, equipment, and storage medium for delay estimation. The related embodiments can be applied to various scenarios such as artificial intelligence, maps, and intelligent transportation to improve the accuracy of delay estimation. The method of the embodiments of the present application includes: obtaining the received point signal collected by the sound receiving device at the current moment, and obtaining the reference point signal output by the sound playback device at the current moment; based on the first delay estimation algorithm, performing delay calculation on the reference point signal and the received point signal to obtain the first delay; based on the second delay estimation algorithm, performing delay calculation on the reference point signal and the received point signal to obtain the second delay, wherein the duration of calculating the second delay is greater than the duration of calculating the first delay; if both the first delay and the second delay are non-empty valid values, then taking the second delay as the candidate delay; determining the target delay based on the candidate delay and the historical delay at the previous moment.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of communication technologies, and in particular, to a method, apparatus, device, and storage medium for delay estimation. Background Art

[0002] With the development of communication technologies, more and more application scenarios involve the call process. During a call, if the sound signal collected by the microphone includes not only the near-end voice signal (generated by the user speaking during the call), but also the far-end voice signal, it will cause echo interference during the call, thereby directly affecting the call quality. To this end, some applications such as acoustic echo cancellation and certain howling suppression can be used to eliminate echoes.

[0003] However, the prerequisite for the successful operation of the echo suppression or howling suppression algorithm is to accurately estimate the echo delay. However, traditional delay estimation techniques usually quantify the signal energies at the reference point (such as the speaker) and the receiving point (such as the microphone) into binary sequences to obtain the energy binary quantization values of the sub-bands, and determine the delay based on the energy binary quantization values of the sub-bands as features. Although traditional delay estimation techniques reduce the computational complexity of delay estimation, due to the poor stability of signal energy and easy fluctuations, the accuracy of delay estimation is reduced. Summary of the Invention

[0004] Embodiments of the present application provide a method, apparatus, device, and storage medium for delay estimation, which are used to calculate the first delay and the second delay by combining the use of a first delay estimation algorithm and a second delay estimation algorithm with different calculation rates and different reliabilities, so as to enhance the stability and reliability of delay estimation, and further be able to combine the delay estimation combination logic to further determine the target delay from the first delay, the second delay, and the historical delay, thereby improving the accuracy of delay estimation.

[0005] On the one hand, embodiments of the present application provide a method for delay estimation, including:

[0006] Obtain the received signal at the receiving point collected by the sound receiving device at the current moment, and obtain the reference signal output by the sound playing device at the current moment;

[0007] Based on the first delay estimation algorithm, perform delay calculation on the reference signal and the received signal to obtain the first delay;

[0008] Based on the second delay estimation algorithm, perform delay calculation on the reference signal and the received signal to obtain the second delay, where the duration of calculating the second delay is longer than the duration of calculating the first delay;

[0009] If both the first delay and the second delay are non-empty valid values, then use the second delay as the candidate delay;

[0010] Determine a target delay based on a candidate delay and a historical delay at the previous moment.

[0011] On the other hand, this application provides a delay estimation device, including:

[0012] An acquisition unit, configured to acquire a received point signal collected by a sound receiving device at the current moment, and acquire a reference point signal output by a sound playback device at the current moment;

[0013] The acquisition unit is further configured to perform a delay calculation on the reference point signal and the received point signal based on a first delay estimation algorithm to obtain a first delay;

[0014] The acquisition unit is further configured to perform a delay calculation on the reference point signal and the received point signal based on a second delay estimation algorithm to obtain a second delay, where the duration of calculating the second delay is greater than the duration of calculating the first delay;

[0015] A processing unit, configured to use the second delay as a candidate delay if both the first delay and the second delay are non-empty valid values;

[0016] A determination unit, configured to determine a target delay based on the candidate delay and a historical delay at the previous moment.

[0017] In a possible design, in an implementation manner of another aspect of the embodiments of this application,

[0018] The processing unit is further configured to use the first delay as a candidate delay if the first delay is a non-empty valid value and the second delay is an empty value or an invalid value.

[0019] In a possible design, in an implementation manner of another aspect of the embodiments of this application, the determination unit may specifically be configured to:

[0020] Calculate a first difference between the candidate delay and a first time threshold;

[0021] If the candidate delay is greater than or equal to the first time threshold, use the first difference as an intermediate candidate delay;

[0022] If the historical delay is a non-empty valid value, determine the target delay based on the historical delay and the intermediate candidate delay;

[0023] If the historical delay is an empty value or an invalid value, use the intermediate candidate delay as the target delay.

[0024] In a possible design, in an implementation manner of another aspect of the embodiments of this application, the determination unit may specifically be configured to:

[0025] Calculate an absolute value of a difference between the historical delay and the intermediate candidate delay;

[0026] If the absolute value of the difference is less than or equal to the first time threshold, the historical delay is used as the target delay;

[0027] If the absolute value of the difference is greater than the first time threshold, the historical delay is compared with the intermediate candidate delay to obtain a comparison result;

[0028] Based on the comparison result, the target delay is determined.

[0029] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the determining unit may specifically be used for:

[0030] If the comparison result is that the historical delay is greater than the intermediate candidate delay, calculate the sum of the intermediate candidate delay and the second time threshold;

[0031] Use the sum as the target delay;

[0032] If the comparison result is that the historical delay is less than the intermediate candidate delay, use the intermediate candidate delay as the target delay.

[0033] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the obtaining unit may specifically be used for:

[0034] Perform downsampling on the reference point signal to obtain a first reference point sampling signal;

[0035] Perform feature extraction on the first reference point sampling signal to obtain reference audio features;

[0036] Use the reference audio features as historical audio features and store them in the historical feature memory, where the historical feature memory is used to store K historical audio features, and K is an integer greater than 1;

[0037] Perform downsampling on the receiving point signal to obtain a first receiving point sampling signal;

[0038] Perform feature extraction on the first receiving point sampling signal to obtain received audio features;

[0039] Match the received audio features with the K historical audio features in the historical feature memory respectively to obtain K matching scores, where one matching score corresponds to one historical audio feature;

[0040] Based on the K matching scores, determine the first delay.

[0041] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the obtaining unit may specifically be used for:

[0042] According to the storage positions of the historical audio features in the historical feature memory, store the K matching scores correspondingly in the matching result memory;

[0043] Perform smoothing statistics on the K matching scores respectively to obtain K smoothed scores;

[0044] According to the storage positions of the K matching scores in the matching result memory, store the K smoothed scores in the matching result smoothing memory;

[0045] Determine the first delay based on the K smoothed scores and the positions of the K smoothed scores in the matching result smoothing memory.

[0046] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the obtaining unit may specifically be used for:

[0047] Select the first smoothed score and the second smoothed score with the smallest numerical values from the K smoothed scores;

[0048] Calculate the second difference between the first smoothed score and the second smoothed score;

[0049] If the second difference is greater than or equal to the score threshold, determine the first delay based on the first smoothed score;

[0050] If the second difference is less than the score threshold, set the first delay to a null value or an invalid value.

[0051] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the obtaining unit may specifically be used for:

[0052] Obtain the time interval between the current moment and the previous moment;

[0053] Obtain the position interval between the first smoothed score and the last smoothed score in the matching result smoothing memory;

[0054] Calculate the first delay based on the position interval and the time interval.

[0055] In a possible design, in an implementation manner of another aspect of the embodiments of the present application, the obtaining unit may specifically be used for:

[0056] Perform downsampling processing on the reference point signal to obtain a second reference point sampling signal;

[0057] Perform downsampling processing on the receiving point signal to obtain a second receiving point sampling signal;

[0058] Perform filtering processing on the second reference point sampling signal and the second receiving point sampling signal to obtain the amplitude value distribution of the filter coefficients;

[0059] Determine a second delay based on the amplitude value distribution of the filter coefficients.

[0060] In a possible design, in an implementation of another aspect of the embodiments of the present application, the obtaining unit may specifically be used for:

[0061] Obtain the distribution position corresponding to the maximum amplitude value in the amplitude value distribution of the filter coefficients;

[0062] Determine the second delay based on the distribution position.

[0063] In a possible design, in an implementation of another aspect of the embodiments of the present application, the obtaining unit may specifically be used for:

[0064] Calculate the average value of all amplitude values between the latest amplitude value and the maximum amplitude value in the amplitude value distribution of the filter coefficients;

[0065] Calculate a third difference between the maximum amplitude value and the average value;

[0066] If the third difference is greater than the amplitude threshold, determine the second delay based on the distribution position and the downsampling rate;

[0067] If the third difference is less than the amplitude threshold, set the second delay to a null value or an invalid value.

[0068] Another aspect of the present application provides a computer device, including: a memory, a processor, and a bus system;

[0069] Wherein, the memory is used to store programs;

[0070] The processor is used to implement the methods in the above aspects when executing the programs in the memory;

[0071] The bus system is used to connect the memory and the processor to enable communication between the memory and the processor.

[0072] Another aspect of the present application provides a computer-readable storage medium, in which instructions are stored, and when the instructions run on a computer, the computer is enabled to execute the methods in the above aspects.

[0073] It can be seen from the above technical solutions that the embodiments of the present application have the following beneficial effects:

[0074] By acquiring the received point signal collected by the sound receiving device at the current moment and the reference point signal output by the sound playing device at the current moment, based on the first delay estimation algorithm, the delay calculation is performed on the reference point signal and the received point signal to obtain the first delay, and based on the second delay estimation algorithm, the delay calculation is performed on the reference point signal and the received point signal to obtain the second delay with a calculation duration greater than the first delay. When both the first delay and the second delay are non-empty valid values, the second delay is used as the candidate delay, and based on the candidate delay and the historical delay at the previous moment, the target delay is determined. Through the above method, it is possible to calculate the first delay and the second delay by combining the first delay estimation algorithm and the second delay estimation algorithm with different calculation rates and different reliabilities, so as to enhance the stability and reliability of the delay estimation. Furthermore, the delay estimation combination logic can be combined to further determine the target delay from the first delay, the second delay, and the historical delay, thereby improving the accuracy of the delay estimation. Description of the Drawings

[0075] Figure 1 It is a schematic architecture diagram of the delay data control system in an embodiment of the present application;

[0076] Figure 2 It is a flowchart of an embodiment of the delay estimation method in an embodiment of the present application;

[0077] Figure 3 It is a flowchart of another embodiment of the delay estimation method in an embodiment of the present application;

[0078] Figure 4 It is a flowchart of another embodiment of the delay estimation method in an embodiment of the present application;

[0079] Figure 5 It is a flowchart of another embodiment of the delay estimation method in an embodiment of the present application;

[0080] Figure 6 It is a flowchart of another embodiment of the delay estimation method in an embodiment of the present application;

[0081] Figure 7 It is a flowchart of another embodiment of the delay estimation method in an embodiment of the present application;

[0082] Figure 8 It is a flowchart of another embodiment of the delay estimation method in an embodiment of the present application;

[0083] Figure 9 It is a flowchart of another embodiment of the delay estimation method in an embodiment of the present application;

[0084] Figure 10 It is a flowchart of another embodiment of the delay estimation method in an embodiment of the present application;

[0085] Figure 11 It is another flowchart of the delay estimation method in the embodiment of the present application;

[0086] Figure 12 It is another flowchart of the delay estimation method in the embodiment of the present application;

[0087] Figure 13 It is another flowchart of the delay estimation method in the embodiment of the present application;

[0088] Figure 14 It is a schematic diagram of the principle flowchart of the delay estimation method in the embodiment of the present application;

[0089] Figure 15 It is a schematic diagram of the principle flowchart of echo cancellation of the delay estimation method in the embodiment of the present application;

[0090] Figure 16 It is a schematic diagram of the principle flowchart of the delay estimation combined logic of the delay estimation method in the embodiment of the present application;

[0091] Figure 17 It is a schematic diagram of the principle flowchart of the first delay estimation algorithm of the delay estimation method in the embodiment of the present application;

[0092] Figure 18 It is a schematic diagram of the historical feature matching module of the delay estimation method in the embodiment of the present application;

[0093] Figure 19 It is a schematic diagram of the matching result memory of the delay estimation method in the embodiment of the present application;

[0094] Figure 20 It is a schematic diagram of the matching result smoothing memory of the delay estimation method in the embodiment of the present application;

[0095] Figure 21 It is a schematic diagram of the principle flowchart of the second delay estimation algorithm of the delay estimation method in the embodiment of the present application;

[0096] Figure 22 It is a schematic diagram of the amplitude value distribution of the delay estimation method in the embodiment of the present application;

[0097] Figure 23 It is a schematic diagram of an embodiment of the delay estimation device in the embodiment of the present application;

[0098] Figure 24 It is a schematic diagram of an embodiment of the computer device in the embodiment of the present application. Detailed implementation manners

[0099] The embodiments of the present application provide a delay estimation method, apparatus, device, and storage medium, which are used to calculate a first delay and a second delay by combining and using a first delay estimation algorithm and a second delay estimation algorithm with different calculation rates, that is, different reliabilities, so as to enhance the stability and reliability of delay estimation. Furthermore, the delay estimation combination logic can be combined to further determine a target delay from the first delay, the second delay, and the historical delay, thereby improving the accuracy of delay estimation.

[0100] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims, and drawings of the present application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "corresponding to" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0101] It can be understood that in the specific implementation of the present application, data related to delay estimation, reference point signals, and receiving point signals, etc. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards in relevant countries and regions.

[0102] It can be understood that, as the delay estimation method disclosed in the present application, it specifically relates to the Intelligent Vehicle Infrastructure Cooperative Systems (IVICS). The following further introduces the Intelligent Vehicle Infrastructure Cooperative Systems. The Intelligent Vehicle Infrastructure Cooperative Systems, abbreviated as the vehicle-road collaborative system, is a development direction of the Intelligent Transportation System (ITS). The vehicle-road collaborative system uses advanced wireless communication and new-generation Internet and other technologies to comprehensively implement dynamic real-time information interaction between vehicles and between vehicles and roads, and on the basis of full-time and full-space dynamic traffic information collection and fusion, carry out vehicle active safety control and road collaborative management, fully realize the effective collaboration of people, vehicles, and roads, ensure traffic safety, improve traffic efficiency, and thus form a safe, efficient, and environmentally friendly road traffic system.

[0103] It can be understood that the delay estimation method disclosed in the present application also involves artificial intelligence (AI) technology. The following further introduces the artificial intelligence technology. Artificial intelligence uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, and is a theory, method, technology, and application system that can perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling the machines to have the functions of perception, reasoning, and decision-making.

[0104] Artificial intelligence technology is an interdisciplinary subject involving a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0105] Secondly, natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can enable effective communication between humans and computers in natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, the research in this field will involve natural language, that is, the language used by people in daily life, so it has a close connection with the research of linguistics. Natural language processing technologies usually include technologies such as text processing, semantic understanding, machine translation, robot question answering, and knowledge graphs.

[0106] Secondly, machine learning (ML) is an interdisciplinary subject involving multiple fields such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rote learning.

[0107] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, smart healthcare, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0108] It should be understood that the delay estimation method provided in this application can be applied to various scenarios, including but not limited to artificial intelligence, maps, intelligent transportation, cloud technology, etc., for completing echo cancellation or echo suppression through delay estimation, so as to be applied to scenarios such as intelligent voice communication, telephone communication, video communication, intelligent voice interaction, intelligent map navigation, etc.

[0109] To solve the above problems, this application proposes a delay estimation method, which is applied to Figure 1 the delay data control system shown in Figure 1 , Figure 1 which is a schematic architecture diagram of the delay data control system in an embodiment of this application. As shown in Figure 1 , the server obtains the received point signal collected by the sound receiving device at the current moment provided by the terminal device, and obtains the reference point signal output by the sound playback device at the current moment. Based on the first delay estimation algorithm, the server performs delay calculation on the reference point signal and the received point signal to obtain the first delay, and based on the second delay estimation algorithm, the server performs delay calculation on the reference point signal and the received point signal to obtain the second delay whose calculation duration is greater than the first delay. When both the first delay and the second delay are non-empty valid values, the second delay is used as the candidate delay, and the target delay is determined based on the candidate delay and the historical delay at the previous moment. Through the above method, it is possible to calculate the first delay and the second delay by combining the first delay estimation algorithm and the second delay estimation algorithm with different calculation rates and different reliabilities, so as to enhance the stability and reliability of delay estimation. Furthermore, it is possible to combine the delay estimation combination logic to further determine the target delay from the first delay, the second delay, and the historical delay, thereby improving the accuracy of delay estimation.

[0110] It can be understood that Figure 1 only one type of terminal device is shown in Figure 1A server is shown, but in an actual scenario, multiple servers can also be involved. Especially in the scenario of multi-model training interaction, the number of servers depends on the actual scenario and is not specifically limited here.

[0111] It should be noted that in this embodiment, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms. The terminal device and the server can be directly or indirectly connected through wired or wireless communication methods, and the terminal device and the server can be connected to form a blockchain network, which is not limited in this application.

[0112] Combined with the above introduction, the delay estimation method in this application will be introduced below. Please refer to Figure 2 One embodiment of the delay estimation method in the embodiment of this application includes:

[0113] In step S101, obtain the received signal at the receiving point collected by the sound collection device at the current moment, and obtain the reference signal output by the sound playback device at the current moment;

[0114] It can be understood that since various signals generated during a call are all audio signals, for the sake of easy distinction, in this embodiment, the audio signal output by the sound playback device (such as a speaker or a horn, etc.) can be defined as a reference signal (or a remote voice signal), and the audio signal collected by the sound collection device (such as a microphone, etc.) can be defined as the received signal at the receiving point (or a sound signal). This received signal at the receiving point not only contains audio signals related to the actual call content. For example, when having a voice call, it is directly spoken by the target object, but may also contain a remote voice signal or other noises, etc.

[0115] Among them, during a call, the received signal at the receiving point will be transmitted between the clients where the two call parties are located. If the received signal at the receiving point contains a reference signal, it may cause echo interference during the call and affect the call quality. Therefore, in order to avoid echo interference during the call, it is necessary to implement echo cancellation technology on the received signal at the receiving point to eliminate the echo component in the received signal at the receiving point, that is, the reference signal.

[0116] For example, as Figure 15As shown in the figure, taking acoustic echo cancellation as an example, before the audio signal is sent to a sound playback device (such as a speaker or a horn, etc.) for playback, it needs to pass through position A. The audio signal at position A (reference point) is generally called the reference point signal. The reference point signal is sent to the sound playback device (such as a speaker or a horn, etc.) for playback through the software and hardware playback logic. After being transmitted through media such as air and entering the sound collection device (such as a microphone, etc.), it reaches position B through the software and hardware collection logic. The signal at position B (receiving point) can be called the receiving point signal. The time taken for the signal to be transmitted from the reference point to the sound playback device (such as a speaker or a horn, etc.) is called the software and hardware playback delay. The delay from the speaker through transmission media such as air to the microphone is called the acoustic path delay. The delay from the sound collection device (such as a microphone, etc.) through the software and hardware collection logic to the receiving point is called the software and hardware collection delay.

[0117] For example, when the target object X and the target object Y are having a voice call, first, an audio signal corresponding to the speech content of the target object X is generated, that is, the near-end voice signal C0. The client where the target object X is located collects the sound signal through the microphone X1, obtains the receiving point signal containing the near-end voice signal C0, and outputs it to the client where the target object Y is located.

[0118] For the target object Y, the client where it is located receives the sound signal and outputs it through the speaker Y1 to form the reference point signal C1. When the target object Y hears the reference point signal C1, an audio signal corresponding to the speech content of the target object Y will be generated accordingly, that is, the near-end voice signal C2. Then, the receiving point signal containing the near-end voice signal C2 is collected by the microphone Y2 and output to the client where the target object X is located.

[0119] In the above call process, it can be understood that the reference point signal C1 is actually formed by the output of the near-end voice signal C0 generated by the speech content of the target object X. If the receiving point signal collected by the microphone Y2 includes not only the near-end voice signal C2 but also the reference point signal C1, when the receiving point signal is output to the client where the target object X is located, it will make the target object X feel that what he has said (that is, the near-end voice signal C0) has come back, that is, an echo is generated during the call process.

[0120] Therefore, in order to eliminate the echo component (that is, the reference point signal) in the sound signal subsequently, the receiving point signal at the current moment will be collected first through a sound collection device (such as a microphone), and the reference point signal at the current moment output by the sound playback device (such as a speaker) will be obtained.

[0121] In step S102, based on the first delay estimation algorithm, the delay of the reference point signal and the receiving point signal is calculated to obtain the first delay;

[0122] In this embodiment, asFigure 14 As shown, by combining the use of two first delay estimation algorithms with different calculation rates and thus different reliabilities (such as Figure 14 the delay estimation algorithm 1 shown illustratively) and a second delay estimation algorithm (such as Figure 14 the delay estimation algorithm 2 shown illustratively), after obtaining the reference point signal and the receiving point signal, the reference point signal and the receiving point signal can be input into the first delay estimation algorithm and the second delay estimation algorithm simultaneously to respectively calculate and obtain a first delay (such as Figure 14 the Tde1 shown illustratively) and a first delay (such as Figure 14 the Tde2 shown illustratively).

[0123] It should be noted that the first delay estimation algorithm and the second delay estimation algorithm can be algorithms based on filter coefficients, audio fingerprints, feature spectrum correlation calculations, etc. It can be understood that the reliability of the second delay estimation algorithm is greater than that of the first delay estimation algorithm, which can be specifically manifested as the time duration for the second delay estimation algorithm to calculate the second delay being longer than the time duration for the first delay estimation algorithm to calculate the first delay, and the accuracy rate of the second delay estimation algorithm to calculate the second delay being greater than the accuracy rate of the first delay estimation algorithm to calculate the first delay, etc.

[0124] It can be understood that the first delay estimation algorithm can adopt an algorithm framework such as Figure 17 shown, and can also be based on other delay estimation algorithms different from the second delay estimation algorithm and adopt an algorithm framework such as Figure 21 shown, or adopt other algorithms, which are not specifically limited here.

[0125] Specifically, as Figure 17 shown, based on the first delay estimation algorithm, delay calculation is performed on the reference point signal and the receiving point signal to obtain a first delay. Specifically, it can be to first perform downsampling processing on the reference point signal through the downsampling module shown illustratively above to obtain a first reference point sampling signal, and then perform feature extraction on the first reference point sampling signal through the feature extraction module shown illustratively above to obtain reference audio features. Then, the reference audio features can be stored in the historical feature memory as historical audio features through the historical feature memory shown illustratively as Figure 17 above. Figure 17 above. Figure 17 shown.

[0126] Meanwhile, the receiving point signal can be subjected to downsampling processing through the downsampling module shown illustratively as Figure 17 below to obtain a first receiving point sampling signal, and then the first receiving point sampling signal can be subjected to feature extraction through the feature extraction module shown illustratively as Figure 17 below to obtain received audio features. Then, through the Figure 17The historical feature matching module shown in the figure will receive the audio features and match them with K historical audio features in the historical feature memory respectively to obtain K matching scores, and determine the first delay based on the K matching scores.

[0127] In step S103, based on the second delay estimation algorithm, the reference point signal and the receiving point signal are subjected to delay calculation to obtain the second delay, wherein the duration of calculating the second delay is greater than the duration of calculating the first delay.

[0128] In this embodiment, as Figure 14 shown, by combining the use of two first delay estimation algorithms with different calculation rates and different reliabilities (such as Figure 14 the delay estimation algorithm 1 shown in the figure) and the second delay estimation algorithm (such as Figure 14 the delay estimation algorithm 2 shown in the figure), after the reference point signal and the receiving point signal are obtained, the reference point signal and the receiving point signal can be input into the first delay estimation algorithm and the second delay estimation algorithm simultaneously to calculate and obtain the first delay (such as Figure 14 Tde1 shown in the figure) and the first delay (such as Figure 14 Tde2 shown in the figure).

[0129] Specifically, as Figure 21 shown, based on the second delay estimation algorithm, the reference point signal and the receiving point signal are subjected to delay calculation to obtain the second delay. Specifically, it can be through the downsampling module shown in Figure 21 the figure above to perform downsampling processing on the reference point signal to obtain the second reference point sampling signal. At the same time, through the downsampling module shown in Figure 21 the figure below to perform downsampling processing on the receiving point signal to obtain the second receiving point sampling signal. Then, the second reference point sampling signal and the second receiving point sampling signal can be filtered through the adaptive system identification filter shown in Figure 21 the figure to obtain the amplitude value distribution of the filter coefficients, and the second delay is determined based on the amplitude value distribution of the filter coefficients. It can be understood that filtering the second reference point sampling signal and the second receiving point sampling signal can not only use the adaptive system identification filter shown in Figure 21 the figure, but also use other filters, or algorithms such as audio fingerprint and feature spectrum correlation calculation, which are not specifically limited here.

[0130] In step S104, if both the first delay and the second delay are non-empty valid values, the second delay is used as the candidate delay.

[0131] In this embodiment, as Figure 14 shown, by combining the use of two first delay estimation algorithms with different calculation rates and different reliabilities (such asFigure 14 The schematic delay estimation algorithm 1) and the second delay estimation algorithm (such as Figure 14 the schematic delay estimation algorithm 2) shown. After obtaining the reference point signal and the receiving point signal, the reference point signal and the receiving point signal can be input into the first delay estimation algorithm and the second delay estimation algorithm simultaneously to calculate the first delay (such as Figure 14 the Tde1) shown and the first delay (such as Figure 14 the Tde2) shown, and the first delay (such as Figure 14 the Tde1) shown and the first delay (such as Figure 14 the Tde2) shown are input into the delay estimation combination logic shown in Figure 14 for delay determination.

[0132] Specifically, as shown in Figure 16 Based on the delay estimation combination logic, the priority between the first delay and the second delay is judged. Assuming that the reliability of the second delay estimation algorithm is higher than that of the first delay estimation algorithm, but the first delay estimation algorithm can calculate an effective Tde1 value, that is, the first delay, relatively quickly. It can be understood that the second delay estimation algorithm has not calculated an effective second delay, that is, the second delay is a null value or an invalid value. Then, the Tde1 value, that is, the first delay, can be defaulted as the candidate delay first, as shown in Figure 16 the first target box "newDelay = Tde1" shown from top to bottom. That is, using Tde1 for newDelay can quickly obtain an effective delay to help perform echo cancellation or echo suppression operations in a timely manner later, and reduce the interference of echo on the voice call.

[0133] Furthermore, when the second delay estimation algorithm can accurately calculate an effective Tde2 value, that is, the second delay, it can be understood that both the first delay and the second delay are non-null and effective values. Then, the second delay can be used as the candidate delay, as shown in Figure 16 the second target box "If Tde2 is a valid value, then newDelay = Tde2" shown from top to bottom. That is, using Tde2 for newDelay can make the current newDelay be assigned to Tde2 as long as the second delay Tde2 is calculated by the second delay estimation algorithm, regardless of whether the first delay Tde1 calculated by the first delay estimation algorithm is a non-null and effective value, or a null value or an invalid value, based on the higher reliability of the second delay estimation algorithm than the first delay estimation algorithm, so as to maintain the accuracy of the delay estimation.

[0134] In step S105, based on the candidate delay and the historical delay at the previous moment, the target delay is determined.

[0135] Specifically, after obtaining the candidate delay, as Figure 16 for the third target box shown from top to bottom, "if newDelay >= T1 ms, then newDelay = newDelay - T1", that is, when the candidate delay newDelay is greater than or equal to the first time threshold T1 ms, for example, when it is greater than or equal to 20 ms, newDelay can be reduced by T1, that is, calculate the first difference between the candidate delay and the first time threshold. This is because generally Figure 15 for the echo cancellation module shown, it prefers to underestimate the delay rather than overestimate the delay. Once the delay is overestimated, the echo cannot be cancelled. Therefore, in this embodiment, the estimated delay can be reduced to leave some margin for the estimation deviation as the intermediate candidate delay, so as to better estimate a more appropriate delay and maintain the accuracy of the delay estimation to a certain extent.

[0136] Further, as Figure 16 for the first diamond target box "Is oldDelay valid?" shown from top to bottom, it can be used to determine whether there is a valid value in the previous delay estimation. It can be understood that when the historical delay is an empty value or an invalid value, that is, oldDelay is invalid, indicating that there has not been a valid delay estimation before, then as Figure 16 for the first target box on the left side of the first diamond target box shown, "Output Tde as newDelay, oldDelay = Tde", that is, the intermediate candidate delay newDelay can be used as the final estimated value Tde, that is, the target delay output, and the historical delay oldDelay is updated with the target delay. When there has not been a valid delay estimation before, the currently obtained intermediate candidate delay can be used as the target delay in time to help perform echo cancellation or echo suppression operations based on the target delay in time later, thereby reducing the interference of the echo on the voice call. On the contrary, when the historical delay is a non-empty valid value, it can be understood that oldDelay already has a valid value, and the target delay is further determined based on the historical delay and the intermediate candidate delay.

[0137] Further, after obtaining the target delay, the reference point signal output by the sound playback device (such as a speaker or a horn, etc.) can be delayed using the target delay, so that the delayed reference point signal is aligned with the reception point collected by the sound collection device (such as a microphone, etc.). Then, the aligned reception point signal and reference point signal are cancelled out each other, thereby realizing the cancellation of the echo component in the sound signal. After the echo cancellation process is completed, the sound signal with the echo component cancelled can be output, that is, transmitted to the client where the other user is located, thus avoiding the echo interference during the call.

[0138] For example, since asFigure 15 The function of the delay estimation module shown is to estimate the delay between the reference point signal and the receiving point signal by comparing them, that is, to determine the target delay. Therefore, after obtaining the target delay, it is possible to calculate whether the error between the target delay Tde and the true delay Td is within an acceptable range, for example, within 80 milliseconds.

[0139] Furthermore, as Figure 15 shown, by using the delay alignment module shown, and using the target delay Tde calculated by the delay estimation module, the signal x_align obtained by delaying the reference point signal x is sent to the echo cancellation module as shown Figure 15 shown. Then, Figure 15 the echo cancellation module shown can use the receiving point signal d and the reference point signal x_align aligned in time to perform echo cancellation.

[0140] In the embodiment of the present application, a delay estimation method is provided. Through the above method, it is possible to calculate the first delay and the second delay by combining the first delay estimation algorithm and the second delay estimation algorithm with different calculation rates and different reliabilities, so as to enhance the stability and reliability of the delay estimation. Furthermore, it is possible to combine the delay estimation combination logic to further determine the target delay from the first delay, the second delay, and the historical delay, thereby improving the accuracy of the delay estimation.

[0141] Optionally, on the basis of the above Figure 2 corresponding embodiment, in another optional embodiment of the delay estimation method provided by the embodiment of the present application, as Figure 3 shown, before determining the target delay based on the candidate delay and the historical delay at the previous moment in step S105, the method further includes:

[0142] In step S301, if the first delay is a non-empty valid value and the second delay is an empty value or an invalid value, then the first delay is used as the candidate delay.

[0143] Specifically, as Figure 16 shown, based on the delay estimation combination logic, the priority between the first delay and the second delay is judged. Assuming that the reliability of the second delay estimation algorithm is higher than that of the first delay estimation algorithm, but the first delay estimation algorithm can calculate an effective Tde1 value, that is, the first delay, relatively quickly. It can be understood that the second delay estimation algorithm has not calculated an effective second delay, that is, the second delay is an empty value or an invalid value. Then, the Tde1 value, that is, the first delay, can be defaulted to be used as the candidate delay, as Figure 16The first target box shown from top to bottom, "newDelay = Tde1", that is, using Tde1 for newDelay, can help perform echo cancellation or echo suppression operations in a timely manner by quickly obtaining an effective delay, reducing the interference of echo on voice calls.

[0144] Optionally, based on the above Figure 2 or Figure 3 In another optional embodiment of the delay estimation method provided by the embodiments of the present application based on the corresponding embodiments, as Figure 4 shown, step S105 determines the target delay based on the candidate delay and the historical delay at the previous moment, including:

[0145] In step S401, calculate the first difference between the candidate delay and the first time threshold;

[0146] In step S402, if the candidate delay is greater than or equal to the first time threshold, then use the first difference as the intermediate candidate delay;

[0147] In step S403, if the historical delay is a non-empty valid value, then determine the target delay based on the historical delay and the intermediate candidate delay;

[0148] In step S404, if the historical delay is an empty value or an invalid value, then use the intermediate candidate delay as the target delay.

[0149] Specifically, after obtaining the candidate delay, as Figure 16 The third target box shown from top to bottom, "If newDelay >= T1 ms, then newDelay = newDelay - T1", that is, when the candidate delay newDelay is greater than or equal to T1 milliseconds, that is, the first time threshold, for example, when it is greater than or equal to 20 ms, newDelay can be reduced by T1, that is, calculate the first difference between the candidate delay and the first time threshold. This is because generally Figure 15 As shown in the echo cancellation module, it prefers an underestimated delay rather than an overestimated delay. Once the delay is overestimated, the echo cannot be cancelled. Therefore, in this embodiment, the estimated delay can be reduced to leave some margin for the estimation deviation as the intermediate candidate delay, so as to better estimate a more appropriate delay and maintain the accuracy of the delay estimation to a certain extent.

[0150] As Figure 16 The first diamond target box shown from top to bottom, "Is oldDelay valid?", can be used to determine whether there is a valid value in the previous delay estimation. It can be understood that when the historical delay is an empty value or an invalid value, that is, oldDelay is invalid, indicating that there has not been a valid delay estimation before, then asFigure 16 The first target box to the left of the first diamond target box shown, "Output Tde as newDelay, oldDelay = Tde", means that the intermediate candidate delay newDelay can be used as the final estimated value Tde, i.e., the target delay output, and the target delay is used to update the historical delay oldDelay. This can timely use the currently obtained intermediate candidate delay as the target delay in the situation where there was no effective delay estimate before, to help perform echo cancellation or echo suppression operations based on the target delay in a timely manner subsequently, thereby reducing the interference of echo on the voice call.

[0151] Conversely, as Figure 16 shown, when the historical delay is a non-empty valid value, it can be understood that there is already a valid value for oldDelay. Then, based on the historical delay and the intermediate candidate delay, the formula in the first target box to the right of the first diamond target box shown as Figure 16 "delayDiff = abs(oldDelay - newDelay)" can be used to calculate the absolute value of the difference delayDiff between the historical delay and the intermediate candidate delay, which can be used to better determine the delay value gap between the intermediate candidate delay and the historical delay based on the absolute value of the difference delayDiff, and whether this gap is within the tolerance range of the echo cancellation filter, so as to further determine whether it is necessary to change the current newDelay to maintain the stability of the delay estimate.

[0152] Furthermore, based on the second diamond target box shown as Figure 16 from top to bottom, "delayDiff > T1ms?", it can be used to determine whether the absolute value of the difference delayDiff is greater than T1 milliseconds, i.e., the first time threshold. When the absolute value of the difference is less than or equal to the first time threshold, it can indicate that the delay value gap between the intermediate candidate delay and the historical delay is not large and is within the tolerance range of the echo cancellation filter. Therefore, to reduce the number of times of changing the delay output (the estimation algorithm often has small estimation deviations, and each change may cause incomplete echo cancellation in a short time), reduce the estimation deviation, and thus maintain the stability and accuracy of the delay estimate, based on the target box connected to the left of the second diamond target box shown as Figure 16 from top to bottom, "Continue to output Tde equal to oldDelay", it can be understood that in this embodiment, the historical delay is continued to be used as the target delay, and the output Tde is maintained equal to the previous estimated value oldDelay of the historical delay. Conversely, when the absolute value of the difference is greater than the first time threshold, for example, greater than 40ms, based on Figure 16"oldDelay > newDelay?", in the target box connected to the right of the second diamond target box shown from top to bottom, is used to further determine the magnitude between the historical delay and the intermediate candidate delay, and based on the determination result, further determine the target delay.

[0153] Optionally, based on the above Figure 4 corresponding embodiment, in another optional embodiment of the delay estimation method provided by the embodiments of the present application, as Figure 5 shown, step S403 determines the target delay based on the historical delay and the intermediate candidate delay, including:

[0154] In step S501, calculate the absolute value of the difference between the historical delay and the intermediate candidate delay;

[0155] In step S502, if the absolute value of the difference is less than or equal to the first time threshold, then use the historical delay as the target delay;

[0156] In step S503, if the absolute value of the difference is greater than the first time threshold, then compare the historical delay with the intermediate candidate delay to obtain a comparison result;

[0157] In step S504, determine the target delay according to the comparison result.

[0158] Specifically, as Figure 16 shown, when the historical delay is a non-empty valid value, it can be understood that oldDelay already has a valid value, then based on the historical delay and the intermediate candidate delay, use the formula in the first target box on the right of the first diamond target box shown as Figure 16 "delayDiff = abs(oldDelay - newDelay)" to calculate the absolute value of the difference delayDiff between the historical delay and the intermediate candidate delay, which can be used to better determine the delay value gap between the intermediate candidate delay and the historical delay based on the absolute value of the difference delayDiff, and whether this gap is within the tolerable range of the echo cancellation filter, so as to further determine whether it is necessary to change the current newDelay to maintain the stability of the delay estimation.

[0159] Furthermore, based on as Figure 16The second diamond target box "delayDiff > T1ms?" shown from top to bottom can be used to determine whether the absolute value of the difference delayDiff is greater than T1 milliseconds, i.e., the first time threshold. When the absolute value of the difference is less than or equal to the first time threshold, it can be used to indicate that the delay value difference between the intermediate candidate delay and the historical delay is not significant and is within the tolerance range of the echo cancellation filter. Therefore, in order to reduce the number of times of changing the delay output (there are often small estimation deviations in the estimation algorithm, and each change may cause incomplete echo cancellation for a short time), reduce the estimation deviation, and thus maintain the stability and accuracy of the delay estimation, based on as Figure 16 In the target box connected to the left of the second diamond target box shown from top to bottom, "continue to output Tde equal to oldDelay", it can be understood that in this embodiment, the historical delay is continued to be used as the target delay, and the output Tde is kept equal to the estimated value oldDelay of the previous historical delay. Conversely, when the absolute value of the difference is greater than the first time threshold, for example, greater than 40ms, based on as Figure 16 The third diamond target box "oldDelay > newDelay?" shown from top to bottom is used to further determine the size between the historical delay and the intermediate candidate delay, that is, to further determine whether the historical delay is greater than the intermediate candidate delay. Specifically, the historical delay and the intermediate candidate delay can be compared, as Figure 16 In the target box connected to the left of the third diamond target box shown from top to bottom, "output Tde as newDelay, oldDelay = Tde", it can be understood that if the comparison result is that the historical delay is greater than the intermediate candidate delay, that is, the intermediate candidate delay newDelay is smaller, then the formula for the final output target delay can be: Tde = newDelay + T2, to calculate an intermediate value between the historical delay and the intermediate candidate delay, so as to better estimate a more appropriate delay and maintain the accuracy of the delay estimation to a certain extent.

[0160] Conversely, if the comparison result is that the historical delay is greater than the intermediate candidate delay, then the intermediate candidate delay is used as the target delay, that is, the output target delay Tde is equal to the intermediate candidate delay newDelay. Finally, the latest obtained target Tde is stored in oldDelay for the next judgment.

[0161] Optionally, on the basis of the above Figure 5 corresponding embodiment, in another optional embodiment of the delay estimation method provided by the embodiment of the present application, as Figure 6 shown, step S504 determines the target delay according to the comparison result, including:

[0162] In step S601, if the comparison result is that the historical delay is greater than the intermediate candidate delay, then calculate the sum value of the intermediate candidate delay and the second time threshold;

[0163] In step S602, use the sum value as the target delay;

[0164] In step S603, if the comparison result is that the historical delay is less than the intermediate candidate delay, then use the intermediate candidate delay as the target delay.

[0165] Specifically, as Figure 16 shown, when the absolute value of the difference is greater than the first time threshold, for example, greater than 40 ms, based on the "oldDelay>newDelay?" in the target box connected to the right side of the second diamond target box shown from top to bottom, it is used to further determine whether the historical delay is greater than the intermediate candidate delay. Specifically, the historical delay can be compared with the intermediate candidate delay. As Figure 16 shown in the target box connected to the left side of the third diamond target box shown from top to bottom, "Output Tde is newDelay, oldDelay = Tde", it can be understood that if the comparison result is that the historical delay is greater than the intermediate candidate delay, that is, the intermediate candidate delay newDelay is smaller. As Figure 16 shown in the target box connected to the right side of the third diamond target box shown from top to bottom, "Output Tde is newDelay + T2, oldDelay = Tde", it can be understood that the final output target delay can use the formula: Tde = newDelay + T2, calculate the sum value between the intermediate candidate delay and the second time threshold T2, that is, calculate an intermediate value between the historical delay and the intermediate candidate delay, so as to better estimate a more appropriate delay and maintain the accuracy of delay estimation to a certain extent. Figure 16 Among them, the second time threshold T2 is less than the first time threshold T1. For example, the second time threshold T2 is equal to 10 ms. When switching between the output first delay Tde1 and the second delay Tde2 of the two delay estimation algorithms, the newly obtained target delay Tde is between Tde1 and Tde2, and it is possible that the absolute values of the differences between the target delay Tde and the first delay Tde1, and between the target delay Tde and the second delay Tde2 are both less than the first time threshold T1. In this way, the update times of the target delay Tde can be reduced, and the stability and accuracy of delay estimation can be maintained. Furthermore, it can help to perform echo cancellation or echo suppression operations in a timely manner subsequently, reduce the interference of echo on voice calls, and reduce the probability of missed echo.

[0166]

[0167] ​Further, if the comparison result shows that the historical delay is greater than the intermediate candidate delay, then the intermediate candidate delay is used as the target delay, that is, the output target delay Tde is equal to the intermediate candidate delay newDelay. Finally, the latest obtained target Tde is stored in oldDelay for use in the next judgment.

[0168] Optionally, based on the above Figure 2 corresponding embodiment, in another optional embodiment of the delay estimation method provided by the embodiments of the present application, as Figure 7 shown, step S102 is based on the first delay estimation algorithm to calculate the delay of the reference point signal and the receiving point signal, and obtain the first delay, including:

[0169] In step S701, downsample the reference point signal to obtain the first reference point sampled signal;

[0170] In step S702, extract features from the first reference point sampled signal to obtain the reference audio features;

[0171] In step S703, store the reference audio features as historical audio features in the historical feature memory, where the historical feature memory is used to store K historical audio features, and K is an integer greater than 1;

[0172] In step S704, downsample the receiving point signal to obtain the first receiving point sampled signal;

[0173] In step S705, extract features from the first receiving point sampled signal to obtain the received audio features;

[0174] In step S706, match the received audio features with the K historical audio features in the historical feature memory respectively to obtain K matching scores, where one matching score corresponds to one historical audio feature;

[0175] In step S707, determine the first delay based on the K matching scores.

[0176] It can be understood that since both the reference point signal and the receiving point signal are digital speech frames, usually each frame is 8 ms, or it can also be 10 ms, 16 ms, 20 ms, etc., which is not limited here. Among them, there may be an overlapping part between two adjacent frames. For example, if it is 8 ms per frame and there is 50% overlap, that is, an 8 ms speech frame is obtained every 4 ms. The first frame is the speech sample from 0 to 8 ms, the second frame is from 4 to 12 ms, the third frame is from 8 to 16 ms, and so on.

[0177] It should be noted that in general voice communication, the digital voice signals at the reference point and the receiving point usually have a sampling rate of 16 kHz or 32 kHz, etc. Taking 32 kHz as an example, according to the Nyquist sampling theorem, the effective bandwidth of the voice collected at a sampling rate of 32 kHz is 16 kHz. Also, since the voice bandwidth required for extracting the audio fingerprint does not need to be so high, because in an actual voice communication system, the frequency range that can effectively characterize the voice is around 300 Hz to 2 kHz, so the actual bandwidth required only needs to be slightly greater than 2 kHz.

[0178] Therefore, in order to reduce the computational amount, in this embodiment, a downsampling module as Figure 17 shown is adopted to downsample the voice frames with a high sampling rate of the reference point signal or the receiving point signal to about 5 kHz. Among them, the effective audio bandwidth corresponding to a sampling rate of 5 kHz is 2.5 kHz.

[0179] Furthermore, a feature extraction module as Figure 17 shown can be adopted, which is responsible for extracting the audio features representing the voice characteristics. The reference audio features extracted from the reference point signal will be stored in a historical feature memory as Figure 17 shown. As Figure 18 shown, the feature value F(n) of each frame after the voice frame of the reference point signal passes through feature extraction is stored at the tail of the historical feature memory.

[0180] Among them, the historical feature memory presets a maximum storage quantity for storing K historical audio features. For example, the maximum storage is 75 frames of features, that is, K = 75. If the features stored in the historical feature memory at the current moment are F(23), F(24),..., F(97) for a total of 75 frames, then the newly extracted feature F(98) at the next moment is put into the tail of the historical feature memory, and the features stored in the historical feature memory are updated to F(24), F(25),..., F(98).

[0181] For example, as Figure 18 shown, for a certain frame of voice signal Q obtained at the reference point, after a transmission delay Td, it reaches the receiving point to obtain a voice frame signal Q'. Due to the influence of distortion and noise in the transmission channel, etc., Q and Q' are different but similar signals. Then, a feature F is extracted from the signal frame Q' at the receiving point. Assuming that the voice frame Q at the reference point is the 80th frame, so the feature obtained by extracting the feature is F(80). After a transmission delay Td, the reference point newly obtains, for example, 31 frames of voice frames, and the 31 calculated feature vectors are sequentially put into the historical feature memory. So when the receiving point obtains the feature F of the voice frame Q', as Figure 18Thirty-one features from F(81) to F(111) are newly added to the historical feature memory as shown. Assuming that the maximum storage of the historical feature memory is 75 frames of features, the current features in the historical feature memory are from F(37) to F(111).

[0182] Further, use the historical feature matching module as shown Figure 17 to sequentially match the received audio features at the current moment from the receiving point with K historical audio features from the reference point to obtain the matching scores for each historical audio feature. Then, use the delay determination strategy module as shown Figure 17 to perform smooth statistics on the K matching scores and store them in the matching result smoothing memory, and further determine the first delay according to the position of the most matching feature in the matching result smoothing memory.

[0183] It can be understood that the audio features extracted by the feature extraction module as shown Figure 17 can be a certain audio energy change index. For example, it is determined whether the energy of a certain frequency band of a speech frame increases by more than 3 dB (it can also be other values) compared to the previous frame. For example, for frequency band 8, if the energy of this frequency band in the previous frame is expressed as Epre dB and the energy of this frequency band in the current frame is expressed as Ecur dB, if Ecur - Epre > 3 is satisfied, the feature of this frequency band is marked as 1, otherwise it is 0. If a total of 32 frequency bands need to be compared, the 0 or 1 value of each frequency band can be stored in one bit of a 32-bit integer. Therefore, the feature value extracted for each frame is exactly a 32-bit integer.

[0184] Optionally, based on the above Figure 7 corresponding embodiment, in another optional embodiment of the delay estimation method provided by the embodiments of the present application, as shown Figure 8 in step S707, determining the first delay based on the K matching scores includes:

[0185] In step S801, store the K matching scores in the matching result memory according to the storage positions of the historical audio features in the historical feature memory;

[0186] In step S802, perform smooth statistics on the K matching scores respectively to obtain K smoothed scores;

[0187] In step S803, store the K smoothed scores in the matching result smoothing memory according to the storage positions of the K matching scores in the matching result memory;

[0188] In step S804, determine the first delay based on the K smoothed scores and the positions of the K smoothed scores in the matching result smoothing memory.

[0189] Specifically, continuing with the example of extracting the feature F from the signal frame Q' at the above-mentioned receiving point and the maximum storage of 75 frames of features in the historical feature memory, it can be as follows Figure 17 As shown in the historical feature matching module, the feature F is respectively matched with these 75 features. Among them, the matching algorithm used can be XOR operation, or other matching algorithms such as regular expressions. There is no specific limitation here, and 75 matching scores can be obtained. For example, S(1), S(2), …, S(75). It can be understood that if the XOR operation is used as the matching algorithm, the more similar two features are, the lower the score.

[0190] Furthermore, the K matching scores calculated by the historical feature matching module as shown in Figure 17 can be stored in the matching result memory as shown in Figure 19 i.e., S(1), S(2), …, S(75). Furthermore, the K matching scores are respectively smoothed and statistically analyzed, and smoothed scores Sm(1), Sm(2), …, Sm(75) can be obtained. Among them, the smoothing statistical method can be common exponential smoothing, or other smoothing algorithms. There is no specific limitation here. Then, according to the storage positions of the K matching scores in the matching result memory, the K smoothed scores are stored in the matching result smoothing memory as shown in Figure 20 i.e., Sm(1), Sm(2), …, Sm(75), and the first delay can be further determined based on the K smoothed scores and the positions of the K smoothed scores in the matching result smoothing memory.

[0191] Optionally, on the basis of the above Figure 8 corresponding embodiment, in another optional embodiment of the delay estimation method provided by the embodiments of the present application, as shown in Figure 9 based on the K smoothed scores and the positions of the K smoothed scores in the matching result smoothing memory, determining the first delay includes:

[0192] In step S901, select the smallest first smoothed score and the second smoothed score from the K smoothed scores;

[0193] In step S902, calculate the second difference between the first smoothed score and the second smoothed score;

[0194] In step S903, if the second difference is greater than or equal to the score threshold, determine the first delay based on the first smoothed score;

[0195] In step S904, if the second difference is less than the score threshold, set the first delay to a null value or an invalid value.

[0196] Specifically, since the smaller the values of the smoothing scores Sm(1), Sm(2), …, Sm(75), the more matching the two features are. Continuing with the example above where 31 new feature values are added to the historical feature memory after the delay Td, then as Figure 18 shown, it can be seen that the feature most matching the feature F is F(80), and the matching score corresponds to S(44) in the matching result memory as shown in Figure 19 shown, and the smoothing score corresponds to Sm(44) in the matching result smoothing memory as shown in Figure 20 shown.

[0197] Furthermore, since the minimum value in Sm(1), Sm(2), …, Sm(75), for example, Sm(44) is not significantly smaller than other values, then the first smoothing score and the second smoothing score with the smallest values can be selected from the K smoothing scores. For example, the first smallest value is the first smoothing score Sm(44) and the second smallest value is the second smoothing score Sm(40). Then, the second difference Sm(40) - Sm(44) between the first smoothing score and the second smoothing score can be calculated. If the second difference is less than the score threshold Threshold1, that is, Sm(40) - Sm(44) < Threshold1, it means that Sm(44) is not significantly small. Therefore, as Figure 17 shown, the delay determination strategy can output an invalid value, for example, "-1001", or output an empty value "Null". On the contrary, if the second difference is greater than the score threshold Threshold1, the first delay can be further determined based on the first smoothing score.

[0198] Optionally, based on the above Figure 9 corresponding embodiment, in another optional embodiment of the delay estimation method provided by the embodiments of the present application, as Figure 10 shown, determining the first delay based on the first smoothing score includes:

[0199] In step S1001, obtain the time interval between the current moment and the previous moment;

[0200] In step S1002, obtain the position interval between the first smoothing score and the last smoothing score in the matching result smoothing memory;

[0201] In step S1003, calculate the first delay based on the position interval and the time interval.

[0202] Specifically, if the second difference is greater than the score threshold Threshold1, it can be obtained that Sm(44) should have the minimum value at this time. Then, the position interval between the first smoothing score and the last smoothing score in the matching result smoothing memory can be obtained. For example, as Figure 20As shown, the position of Sm(44) is exactly 31 frames away from the end of the memory, i.e., the position of Sm(75).

[0203] Furthermore, it can be achieved by, for example, Figure 20 As shown, the position of the minimum smoothing score in the memory can be smoothed by the matching result to obtain the first delay Tde1. For example, assume that the first delay estimation algorithm calculates the first delay as 31 speech frames. At the same time, the time interval between the current moment and the previous moment can be obtained. For example, each speech frame takes 4 ms to reach the next speech frame. Then, the first delay is calculated based on the position interval and the time interval. For example, the first delay Tde1 is equal to 31 * 4 ms, which is 124 ms.

[0204] Optionally, based on the above Figure 2 corresponding embodiment, in another optional embodiment of the delay estimation method provided by the embodiments of the present application, as Figure 11 shown, step S103 is based on the second delay estimation algorithm to calculate the delay of the reference point signal and the receiving point signal to obtain the second delay, including:

[0205] In step S1101, downsample the reference point signal to obtain the second reference point sampling signal;

[0206] In step S1102, downsample the receiving point signal to obtain the second receiving point sampling signal;

[0207] In step S1103, filter the second reference point sampling signal and the second receiving point sampling signal to obtain the amplitude value distribution of the filter coefficients;

[0208] In step S1104, determine the second delay based on the amplitude value distribution of the filter coefficients.

[0209] Specifically, as Figure 21 shown, based on the second delay estimation algorithm, calculate the delay of the reference point signal and the receiving point signal to obtain the second delay. Specifically, it can be achieved by, for example, Figure 21 the downsampling module shown above to downsample the reference point signal to obtain the second reference point sampling signal. At the same time, by, for example, Figure 21 the downsampling module shown below to downsample the receiving point signal to obtain the second receiving point sampling signal. Then, it can be achieved by, for example, Figure 21 the adaptive system identification filter shown to filter the second reference point sampling signal and the second receiving point sampling signal to obtain the amplitude value distribution of the filter coefficients.

[0210] For example, as Figure 21As shown in the figure, in this embodiment, a standard system identification adaptive LMS filter can be adopted. The sampling rate is reduced by a downsampling module, and only the low-frequency band with more voice signal energy is retained, for example, below 2 kHz, so as to save the amount of calculation.

[0211] Further, as Figure 22 shown, after the reference point signal and the receiving point signal are continuously input, the filter gradually converges through the LMS algorithm to obtain the acoustic frequency response from the reference point to the receiving point. The peak value can appear at the delay point through the amplitude value of the filter coefficient to obtain the amplitude value distribution of the filter coefficient. Then, based on the amplitude value distribution of the filter coefficient, the second delay can be further determined.

[0212] Optionally, based on the above Figure 11 corresponding embodiment, in another optional embodiment of the delay estimation method provided by the embodiment of the present application, as Figure 12 shown, step S1104 determines the second delay based on the amplitude value distribution of the filter coefficient, including:

[0213] In step S1201, obtain the distribution position corresponding to the maximum amplitude value in the amplitude value distribution of the filter coefficient;

[0214] In step S1202, determine the second delay based on the distribution position.

[0215] Specifically, as Figure 22 shown, after obtaining the amplitude value distribution of the filter coefficient, the distribution position corresponding to the maximum amplitude value can be directly obtained from the amplitude value distribution of the filter coefficient. For example, the distribution position corresponding to the maximum amplitude value is the 300th filter coefficient.

[0216] Further, the average value of all amplitude values between the latest amplitude value and the maximum amplitude value in the amplitude value distribution of the filter coefficient can be calculated, and the third difference between the maximum amplitude value and the average value can be calculated. Then, if the third difference is greater than the amplitude threshold, it can indicate that the distribution position where the maximum amplitude value is located represents the second delay, and the second delay can be calculated based on the distribution position and the downsampling rate; otherwise, when the third difference is less than the amplitude threshold, it indicates that there is no second delay currently, and the second delay can be set to a null value or an invalid value.

[0217] Optionally, based on the above Figure 12 corresponding embodiment, in another optional embodiment of the delay estimation method provided by the embodiment of the present application, as Figure 13 shown, step S1202 determines the second delay based on the distribution position, including:

[0218] In step S1301, calculate the average value of all amplitude values between the latest amplitude value and the maximum amplitude value in the amplitude value distribution of the filter coefficients;

[0219] In step S1302, calculate the third difference between the maximum amplitude value and the average value;

[0220] In step S1303, if the third difference is greater than the amplitude threshold, calculate the second delay based on the distribution position and the downsampling rate;

[0221] In step S1304, if the third difference is less than the amplitude threshold, set the second delay to a null value or an invalid value.

[0222] Specifically, after obtaining the distribution position corresponding to the maximum amplitude value in the amplitude value distribution of the filter coefficients, at this time, as Figure 22 shown, the delay determination strategy is to detect the maximum value point of the filter coefficient amplitude. For example, as Figure 22 shown, calculate the third difference between the maximum amplitude value and the average value. If the third difference is greater than the amplitude threshold, that is, the amplitude value of the maximum value point is greater than the average value of all the values on the left by threshold2 dB, which is the amplitude threshold, it can be considered that the position of the maximum value represents the delay value.

[0223] For example, if the maximum amplitude value is the 300th coefficient, and the sampling rate after downsampling of the audio signal is 4 kHz (corresponding to a 2 kHz bandwidth), then each sampling point corresponds to 1 / 4000 Hz seconds, that is, 0.25 ms. Therefore, calculating the second delay based on the distribution position and the downsampling rate can be expressed as 300 * 0.25 = 75 ms.

[0224] Conversely, if the third difference is less than the amplitude threshold, that is, the amplitude value of the maximum value point is less than the average value of all the values on the left by less than threshold2 dB, which is the amplitude threshold, then the second delay can be set to a null value or an invalid value. That is, at this time, as Figure 22 shown, the delay determination strategy outputs the second delay as an invalid value, for example, "-1001", or outputs the second delay as a null value, for example, "Null".

[0225] The delay estimation device in the present application will be described in detail below. Please refer to Figure 23 , Figure 23 which is a schematic diagram of an embodiment of the delay estimation device in the embodiment of the present application. The delay estimation device 20 includes:

[0226] An acquisition unit 201, configured to acquire the received point signal collected by the sound receiving device at the current moment, and acquire the reference point signal output by the sound playing device at the current moment;

[0227] The obtaining unit 201 is further configured to calculate the delay of the reference point signal and the receiving point signal based on the first delay estimation algorithm, and obtain the first delay.

[0228] The obtaining unit 201 is further configured to calculate the delay of the reference point signal and the receiving point signal based on the second delay estimation algorithm, and obtain the second delay, where the duration of calculating the second delay is greater than the duration of calculating the first delay.

[0229] The processing unit 202 is configured to use the second delay as the candidate delay if both the first delay and the second delay are non-empty valid values.

[0230] The determining unit 203 is configured to determine the target delay based on the candidate delay and the historical delay at the previous moment.

[0231] Optionally, based on the above Figure 23 corresponding embodiment, in another embodiment of the delay estimation device provided by the embodiment of the present application,

[0232] The processing unit 202 is further configured to use the first delay as the candidate delay if the first delay is a non-empty valid value and the second delay is an empty value or an invalid value.

[0233] Optionally, based on the above Figure 23 corresponding embodiment, in another embodiment of the delay estimation device provided by the embodiment of the present application, the determining unit 203 may specifically be configured to:

[0234] Calculate a first difference between the candidate delay and a first time threshold;

[0235] If the candidate delay is greater than or equal to the first time threshold, use the first difference as the intermediate candidate delay;

[0236] If the historical delay is a non-empty valid value, determine the target delay based on the historical delay and the intermediate candidate delay;

[0237] If the historical delay is an empty value or an invalid value, use the intermediate candidate delay as the target delay.

[0238] Optionally, based on the above Figure 23 corresponding embodiment, in another embodiment of the delay estimation device provided by the embodiment of the present application, the determining unit 203 may specifically be configured to:

[0239] Calculate the absolute value of the difference between the historical delay and the intermediate candidate delay;

[0240] If the absolute value of the difference is less than or equal to the first time threshold, use the historical delay as the target delay;

[0241] If the absolute value of the difference is greater than the first time threshold, compare the historical delay with the intermediate candidate delay to obtain a comparison result;

[0242] Determine the target delay according to the comparison result.

[0243] Optionally, based on the corresponding embodiment above, in another embodiment of the delay estimation device provided by the embodiments of the present application, the determination unit 203 may specifically be used for: Figure 23

[0244] If the comparison result is that the historical delay is greater than the intermediate candidate delay, calculate the sum of the intermediate candidate delay and the second time threshold;

[0245] Take the sum as the target delay;

[0246] If the comparison result is that the historical delay is less than the intermediate candidate delay, take the intermediate candidate delay as the target delay.

[0247] Figure 23 Optionally, based on the corresponding embodiment above, in another embodiment of the delay estimation device provided by the embodiments of the present application, the acquisition unit 201 may specifically be used for:

[0248] Perform downsampling processing on the reference point signal to obtain a first reference point sampling signal;

[0249] Perform feature extraction on the first reference point sampling signal to obtain reference audio features;

[0250]

[0251] Store the reference audio features as historical audio features in a historical feature memory, where the historical feature memory is used to store K historical audio features, and K is an integer greater than 1;

[0252] Perform downsampling processing on the receiving point signal to obtain a first receiving point sampling signal;

[0253] Perform feature extraction on the first receiving point sampling signal to obtain received audio features;

[0253] Match the received audio features with the K historical audio features in the historical feature memory respectively to obtain K matching scores, where one matching score corresponds to one historical audio feature;

[0254] Determine the first delay based on the K matching scores.

[0255] Optionally, based on the corresponding embodiment above, in another embodiment of the delay estimation device provided by the embodiments of the present application, the acquisition unit 201 may specifically be used for: Figure 23

[0256] ​According to the storage positions of the historical audio features in the historical feature memory, store the K matching scores correspondingly in the matching result memory;

[0257] Perform smoothing statistics on the K matching scores respectively to obtain K smoothed scores;

[0258] According to the storage positions of the K matching scores in the matching result memory, store the K smoothed scores in the matching result smoothing memory;

[0259] Determine the first delay based on the K smoothed scores and the positions of the K smoothed scores in the matching result smoothing memory.

[0260] Optionally, on the basis of the above Figure 23 corresponding embodiment, in another embodiment of the delay estimation device provided by the embodiments of the present application, the obtaining unit 201 may specifically be used for:

[0261] Select the first smoothed score and the second smoothed score with the smallest values from the K smoothed scores;

[0262] Calculate the second difference between the first smoothed score and the second smoothed score;

[0263] If the second difference is greater than or equal to the score threshold, determine the first delay based on the first smoothed score;

[0264] If the second difference is less than the score threshold, set the first delay to a null value or an invalid value.

[0265] Optionally, on the basis of the above Figure 23 corresponding embodiment, in another embodiment of the delay estimation device provided by the embodiments of the present application, the obtaining unit 201 may specifically be used for:

[0266] Obtain the time interval between the current moment and the previous moment;

[0267] Obtain the position interval between the first smoothed score and the last smoothed score in the matching result smoothing memory;

[0268] Calculate the first delay based on the position interval and the time interval.

[0269] Optionally, on the basis of the above Figure 23 corresponding embodiment, in another embodiment of the delay estimation device provided by the embodiments of the present application, the obtaining unit 201 may specifically be used for:

[0270] Perform downsampling on the reference point signal to obtain a second reference point sampling signal;

[0271] Perform downsampling on the receiving point signal to obtain a second receiving point sampling signal;

[0272] Filter the sampling signal of the second reference point and the sampling signal of the second receiving point to obtain the amplitude value distribution of the filter coefficients;

[0273] Determine the second delay based on the amplitude value distribution of the filter coefficients.

[0274] Optionally, based on the corresponding embodiment above, in another embodiment of the delay estimation device provided by the embodiments of the present application, the obtaining unit 201 may specifically be used for: Figure 23

[0275] Obtain the distribution position corresponding to the maximum amplitude value in the amplitude value distribution of the filter coefficients;

[0276] Determine the second delay based on the distribution position.

[0277] Optionally, based on the corresponding embodiment above, in another embodiment of the delay estimation device provided by the embodiments of the present application, the obtaining unit 201 may specifically be used for: Figure 23

[0278] Calculate the average value of all amplitude values between the latest amplitude value and the maximum amplitude value in the amplitude value distribution of the filter coefficients;

[0279] Calculate the third difference between the maximum amplitude value and the average value;

[0280] If the third difference is greater than the amplitude threshold, calculate the second delay based on the distribution position and the downsampling rate;

[0281] If the third difference is less than the amplitude threshold, set the second delay to a null value or an invalid value.

[0282] On the other hand, the present application provides a schematic diagram of another computer device, as Figure 24 shown Figure 24 ​​It is a schematic structural diagram of a computer device provided by an embodiment of the present application. The computer device 300 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPUs) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage devices) for storing application programs 331 or data 332. Among them, the memory 320 and the storage media 330 may be transient storage or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the computer device 300. Further, the central processor 310 may be configured to communicate with the storage media 330 and execute a series of instruction operations in the storage media 330 on the computer device 300.

[0283] The computer device 300 may further include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 333, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM and so on.

[0284] The above-mentioned computer device 300 is also used to execute the steps in the Figures 2 to 13 corresponding embodiment.

[0285] On the other hand, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the method described in the Figures 2 to 13 illustrated embodiment are implemented.

[0286] On the other hand, the present application provides a computer program product including a computer program, and when the computer program is executed by a processor, the steps in the method described in the Figures 2 to 13 illustrated embodiment are implemented.

[0287] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0288] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical, or other forms.

[0289] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0290] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0291] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs and other various media that can store program codes.

Claims

1. A delay estimation method, characterized in that, it includes: obtaining the received point signal collected by the sound receiving device at the current moment, and obtaining the reference point signal output by the sound playing device at the current moment; performing delay calculation on the reference point signal and the received point signal based on the first delay estimation algorithm to obtain a first delay; performing delay calculation on the reference point signal and the received point signal based on the second delay estimation algorithm to obtain a second delay, wherein the duration for calculating the second delay is longer than the duration for calculating the first delay; if both the first delay and the second delay are non-empty valid values, then using the second delay as the candidate delay; determining the target delay based on the candidate delay and the historical delay at the previous moment; wherein, the determining the target delay based on the candidate delay and the historical delay at the previous moment includes: calculating a first difference between the candidate delay and the first time threshold; if the candidate delay is greater than or equal to the first time threshold, then using the first difference as the intermediate candidate delay; if the historical delay is a non-empty valid value, then determining the target delay based on the historical delay and the intermediate candidate delay; if the historical delay is an empty value or an invalid value, then using the intermediate candidate delay as the target delay.

2. The method according to claim 1, characterized in that, before the determining the target delay based on the candidate delay and the historical delay at the previous moment, the method further includes: if the first delay is a non-empty valid value and the second delay is an empty value or an invalid value, then using the first delay as the candidate delay.

3. The method according to claim 1, characterized in that, the determining the target delay based on the historical delay and the intermediate candidate delay includes: calculating the absolute value of the difference between the historical delay and the intermediate candidate delay; if the absolute value of the difference is less than or equal to the first time threshold, then using the historical delay as the target delay; if the absolute value of the difference is greater than the first time threshold, then comparing the historical delay with the intermediate candidate delay to obtain a comparison result; determining the target delay according to the comparison result.

4. The method according to claim 3, characterized in that, the determining the target delay according to the comparison result includes: if the comparison result is that the historical delay is greater than the intermediate candidate delay, then calculating the sum value of the intermediate candidate delay and the second time threshold; using the sum value as the target delay; if the comparison result is that the historical delay is less than the intermediate candidate delay, then using the intermediate candidate delay as the target delay.

5. The method according to claim 1, characterized in that, the performing delay calculation on the reference point signal and the received point signal based on the first delay estimation algorithm to obtain a first delay includes: performing downsampling processing on the reference point signal to obtain a first reference point sampling signal; performing feature extraction on the first reference point sampling signal to obtain reference audio features; Store the reference audio feature as a historical audio feature in a historical feature memory, where the historical feature memory is used to store K historical audio features, and K is an integer greater than 1; Perform downsampling on the received point signal to obtain a first received point sampling signal; Extract features from the first received point sampling signal to obtain a received audio feature; Match the received audio feature with the K historical audio features in the historical feature memory respectively to obtain K matching scores, where one matching score corresponds to one of the historical audio features; Determine the first delay based on the K matching scores.

6. The method according to claim 5, wherein, the determining the first delay based on the K matching scores includes: Correspondingly store the K matching scores in a matching result memory according to the storage positions of the historical audio features in the historical feature memory; Perform smoothing statistics on the K matching scores respectively to obtain K smoothed scores; Store the K smoothed scores in a matching result smoothing memory according to the storage positions of the K matching scores in the matching result memory; Determine the first delay based on the K smoothed scores and the positions of the K smoothed scores in the matching result smoothing memory.

7. The method according to claim 6, wherein, the determining the first delay based on the K smoothed scores and the positions of the K smoothed scores in the matching result smoothing memory includes: Select a first smoothed score and a second smoothed score with the smallest values from the K smoothed scores; Calculate a second difference between the first smoothed score and the second smoothed score; If the second difference is greater than or equal to a score threshold, determine the first delay based on the first smoothed score; If the second difference is less than the score threshold, set the first delay to a null value or an invalid value.

8. The method according to claim 7, wherein, the determining the first delay based on the first smoothed score includes: Obtain the time interval between the current moment and the previous moment; Obtain the position interval between the first smoothed score and the last smoothed score in the matching result smoothing memory; Calculate the first delay based on the position interval and the time interval.

9. The method according to claim 1, wherein, the calculating a second delay by performing delay calculation on the reference point signal and the received point signal based on a second delay estimation algorithm includes: Perform downsampling on the reference point signal to obtain a second reference point sampling signal; Perform downsampling on the received point signal to obtain a second received point sampling signal; Perform filtering on the second reference point sampling signal and the second received point sampling signal to obtain the amplitude value distribution of filter coefficients; Determine the second delay based on the amplitude value distribution of the filter coefficients.

10. The method according to claim 9, wherein, the determining the second delay based on the amplitude value distribution of the filter coefficients includes: Obtain the distribution position corresponding to the maximum amplitude value in the amplitude value distribution of the filter coefficients; Determine the second delay based on the distribution position.

11. The method according to claim 10, wherein, the determining the second delay based on the distribution position includes: Calculate the average value of all amplitude values between the latest amplitude value and the maximum amplitude value in the amplitude value distribution of the filter coefficients; Calculate the third difference between the maximum amplitude value and the average value; If the third difference is greater than the amplitude threshold, calculate the second delay based on the distribution position and the downsampling rate; If the third difference is less than the amplitude threshold, set the second delay to a null value or an invalid value.

12. A delay estimation device, wherein, it includes: An acquisition unit, configured to acquire the received point signal collected by the sound receiving device at the current moment, and acquire the reference point signal output by the sound playback device at the current moment; The acquisition unit is further configured to perform delay calculation on the reference point signal and the received point signal based on a first delay estimation algorithm to obtain a first delay; The acquisition unit is further configured to perform delay calculation on the reference point signal and the received point signal based on a second delay estimation algorithm to obtain a second delay, wherein the duration of calculating the second delay is greater than the duration of calculating the first delay; A processing unit, configured to use the second delay as a candidate delay if both the first delay and the second delay are non-null valid values; A determination unit, configured to determine a target delay based on the candidate delay and the historical delay at the previous moment; Specifically, the determination unit may be configured to: Calculate a first difference between the candidate delay and the first time threshold; If the candidate delay is greater than or equal to the first time threshold, use the first difference as an intermediate candidate delay; If the historical delay is a non-null valid value, determine the target delay based on the historical delay and the intermediate candidate delay; If the historical delay is a null value or an invalid value, use the intermediate candidate delay as the target delay.

13. The device according to claim 12, wherein, the processing unit is further configured to use the first delay as the candidate delay if the first delay is a non-null valid value and the second delay is a null value or an invalid value.

14. The device according to claim 12, wherein, Specifically, the determination unit may be configured to: Calculate the absolute value of the difference between the historical delay and the intermediate candidate delay; If the absolute value of the difference is less than or equal to the first time threshold, use the historical delay as the target delay; If the absolute value of the difference is greater than the first time threshold, compare the historical delay with the intermediate candidate delay to obtain a comparison result; Determine the target delay according to the comparison result.

15. The device according to claim 14, wherein, Specifically, the determination unit may be configured to: If the comparison result is that the historical delay is greater than the intermediate candidate delay, calculate the sum value of the intermediate candidate delay and the second time threshold; Use the sum value as the target delay; If the comparison result is that the historical delay is less than the intermediate candidate delay, use the intermediate candidate delay as the target delay.

16. The device according to claim 12, characterized in that, The obtaining unit may specifically be configured to: Perform downsampling processing on the reference point signal to obtain a first reference point sampling signal; Extract features from the first reference point sampling signal to obtain reference audio features; Use the reference audio features as historical audio features and store them in a historical feature memory, where the historical feature memory is used to store K historical audio features, and K is an integer greater than 1; Perform downsampling processing on the receiving point signal to obtain a first receiving point sampling signal; Extract features from the first receiving point sampling signal to obtain received audio features; Match the received audio features with the K historical audio features in the historical feature memory respectively to obtain K matching scores, where one matching score corresponds to one of the historical audio features; Determine the first delay based on the K matching scores.

17. The device according to claim 16, characterized in that, The obtaining unit may specifically be configured to: According to the storage position of the historical audio features in the historical feature memory, store the K matching scores correspondingly in a matching result memory; Perform smoothing statistics on the K matching scores respectively to obtain K smoothed scores; According to the storage position of the K matching scores in the matching result memory, store the K smoothed scores in a matching result smoothing memory; Determine the first delay based on the K smoothed scores and the positions of the K smoothed scores in the matching result smoothing memory.

18. The device according to claim 17, characterized in that, The obtaining unit may specifically be configured to: Select a first smoothed score and a second smoothed score with the smallest numerical values from the K smoothed scores; Calculate a second difference between the first smoothed score and the second smoothed score; If the second difference is greater than or equal to a score threshold, determine the first delay based on the first smoothed score; If the second difference is less than the score threshold, set the first delay to a null value or an invalid value.

19. The device according to claim 18, characterized in that, The obtaining unit may specifically be configured to: Obtain the time interval between the current moment and the previous moment; Obtain the position interval between the first smoothed score and the last smoothed score in the matching result smoothing memory; Calculate the first delay based on the position interval and the time interval.

20. The device according to claim 12, characterized in that, The obtaining unit may specifically be configured to: Perform downsampling processing on the reference point signal to obtain a second reference point sampling signal; Perform downsampling processing on the receiving point signal to obtain a second receiving point sampling signal; Perform filtering processing on the second reference point sampling signal and the second receiving point sampling signal to obtain the amplitude value distribution of the filter coefficients; Determine the second delay based on the amplitude value distribution of the filter coefficients.

21. The apparatus according to claim 20, wherein, the obtaining unit may specifically be configured to: obtain the distribution position corresponding to the maximum amplitude value in the amplitude value distribution of the filter coefficients; determine the second delay based on the distribution position.

22. The apparatus according to claim 21, wherein, the obtaining unit may specifically be configured to: calculate the average value of all amplitude values between the latest amplitude value and the maximum amplitude value in the amplitude value distribution of the filter coefficients; calculate a third difference between the maximum amplitude value and the average value; if the third difference is greater than the amplitude threshold, calculate the second delay based on the distribution position and the downsampling rate; if the third difference is less than the amplitude threshold, set the second delay to a null value or an invalid value.

23. A computer device, comprising a memory, a processor, and a bus system, where the memory stores a computer program, wherein, when the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented; the bus system is used to connect the memory and the processor to enable communication between the memory and the processor.

24. A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

25. A computer program product, comprising a computer program, wherein, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images