Echo cancellation method, speech recognition method, speech wake-up method and device

Abstract

The embodiments of the present application provide an echo cancellation method, a speech recognition method, a speech wake-up method and a device. The echo cancellation method includes: obtaining original audio data corresponding to the audio data collected by an audio collection unit and the data output by an audio output unit; delaying the original audio data based on the audio data to determine corresponding reference data; determining a transfer parameter corresponding to the audio data, and shortening the audio data based on the transfer parameter to obtain processed data; performing echo cancellation on the processed data based on the reference data to obtain echo-cancelled data after cancellation; performing post-filtering on the echo-cancelled data to obtain corresponding output data; the computational complexity of the echo cancellation process can be reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an echo cancellation method, an echo cancellation device, a speech recognition method, a speech recognition device, a speech wake-up method, a speech wake-up device, an electronic device and a storage medium. Background Art

[0002] In the field of speech recognition, the speech data collected by the device may contain user audio, reverberation generated by the user audio, echo of audio played by the device, reverberation generated by the echo of audio played by the device, etc. Before recognizing the speech data, an echo cancellation method is usually required to eliminate the echo and reverberation.

[0003] An existing echo cancellation method is to use an adaptive filter to perform echo cancellation, wherein the adaptive filter is a device that automatically adjusts the filter coefficients based on the estimation of the statistical characteristics of the input and output signals and adopts a specific algorithm to achieve better filtering characteristics. Specifically, the existing echo cancellation method can spectrally match the spectrum corresponding to the audio and the spectrum corresponding to the original audio, align the audio and the original audio, and input the aligned data into the adaptive filter. The existing adaptive filter is usually an adaptive filter based on the recursive least squares (RLS) algorithm. The adaptive filter continuously adjusts the coefficients of the adaptive filter based on the input data and the output data to estimate the echo data and the reverberation data to offset the echo and reverberation in the voice data.

[0004] However, using the above method, the adaptive filter needs to filter the echo data and reverberation data in the speech audio, which has high computational complexity. Summary of the invention

[0005] The embodiment of the present application provides an echo cancellation method to reduce computational complexity.

[0006] Correspondingly, an embodiment of the present application also provides an echo elimination device, a speech recognition method, a speech recognition device, a speech wake-up method, a speech wake-up device, an electronic device and a storage medium to ensure the implementation and application of the above-mentioned system.

[0007] In order to solve the above problems, an embodiment of the present application discloses an echo cancellation method, which includes: obtaining original audio data corresponding to the audio data collected by an audio acquisition unit and the data output by an audio output unit; based on the audio data, delaying the original audio data to determine corresponding reference data; determining a transfer parameter corresponding to the audio data, and shortening the audio data based on the transfer parameter to obtain processed data; performing echo cancellation on the processed data based on the reference data to obtain echo-cancelled data after cancellation; and performing post-filtering on the echo-cancelled data to obtain corresponding output data.

[0008] In order to solve the above problems, an embodiment of the present application discloses a voice wake-up method, including: obtaining original audio data corresponding to the audio data collected by an audio acquisition unit and the data output by an audio output unit; based on the audio data, delaying the original audio data to determine corresponding reference data; determining a transfer parameter corresponding to the audio data, and shortening the audio data based on the transfer parameter to obtain processed data; performing echo cancellation on the processed data based on the reference data to obtain echo cancellation data after cancellation; performing post-filtering on the echo cancellation data to obtain corresponding output audio data; and waking up the corresponding target object based on the output audio data.

[0009] In order to solve the above problems, an embodiment of the present application discloses a speech recognition method, including: obtaining original audio data corresponding to the audio data collected by an audio acquisition unit and the data output by an audio output unit; based on the audio data, delaying the original audio data to determine corresponding reference data; determining a transfer parameter corresponding to the audio data, and shortening the audio data based on the transfer parameter to obtain processed data; performing echo cancellation on the processed data based on the reference data to obtain echo-cancelled data after cancellation; performing post-filtering on the echo-cancelled data to obtain corresponding output audio data; performing speech recognition on the audio data to determine text information; performing semantic recognition on the text information to determine key information; determining the corresponding target object based on the key information, and performing corresponding processing on the target object.

[0010] In order to solve the above problems, an embodiment of the present application discloses an echo cancellation method, which is applied to a mobile terminal, and the method includes: obtaining original audio data to be output by an audio output unit of the mobile terminal and writing it into a cache; in the process of the audio output unit of the mobile terminal outputting data, obtaining audio data collected by an audio collection unit of the mobile terminal and original audio data of an audio data unit; based on the audio data, delaying the original audio data to determine corresponding reference data; determining a transfer parameter corresponding to the audio data, and shortening the audio data based on the transfer parameter to obtain processed data; performing echo cancellation on the processed data based on the reference data to obtain echo-cancelled data after cancellation; performing post-filtering on the echo-cancelled data to obtain corresponding output data.

[0011] In order to solve the above problems, an embodiment of the present application discloses an echo cancellation device, including: an audio data acquisition module, used to acquire original audio data corresponding to the audio data collected by an audio acquisition unit and the data output by an audio output unit; a reference data acquisition module, used to delay the original audio data based on the audio data and determine the corresponding reference data; a processed data acquisition module, used to determine the transfer parameters corresponding to the audio data, and perform transfer parameter shortening processing on the audio data based on the transfer parameters to obtain processed data; an echo cancellation data acquisition module, used to perform echo cancellation on the processed data based on the reference data to obtain echo cancellation data after cancellation; and an output data acquisition module, used to perform post-filtering processing on the echo cancellation data to obtain corresponding output data.

[0012] In order to solve the above problems, an embodiment of the present application discloses a voice wake-up device, including: an audio data determination module, which is used to obtain the original audio data corresponding to the audio data collected by the audio acquisition unit and the data output by the audio output unit; a reference data determination module, which is used to delay the original audio data based on the audio data to determine the corresponding reference data; a processed data determination module, which is used to determine the transfer parameters corresponding to the audio data, and shorten the audio data based on the transfer parameters to obtain processed data; an echo cancellation data determination module, which is used to perform echo cancellation on the processed data based on the reference data to obtain the echo cancellation data after cancellation; an output data determination module, which is used to perform post-filtering on the echo cancellation data to obtain the corresponding output audio data; a target object wake-up module, which is used to wake up the corresponding target object based on the output audio data.

[0013] In order to solve the above problems, an embodiment of the present application discloses a speech recognition device, including: an audio data acquisition module, which is used to obtain the original audio data corresponding to the audio data collected by the audio acquisition unit and the data output by the audio output unit; a reference data acquisition module, which is used to delay the original audio data according to the audio data and determine the corresponding reference data; a processed data acquisition module, which is used to determine the transfer parameters corresponding to the audio data, and shorten the transfer parameters of the audio data according to the transfer parameters to obtain processed data; an echo cancellation data acquisition module, which is used to perform echo cancellation on the processed data according to the reference data to obtain the echo cancellation data after cancellation; an output data acquisition module, which is used to perform post-filtering processing on the echo cancellation data to obtain the corresponding output audio data; a speech recognition module, which is used to perform speech recognition on the data audio data to determine text information; a semantic recognition module, which is used to perform semantic recognition on the text information to determine key information; a target object processing module, which is used to determine the corresponding target object according to the key information, and perform corresponding processing on the target object.

[0014] In order to solve the above problems, an embodiment of the present application discloses an echo cancellation device, including: an original data cache module, used to obtain the original audio data to be output by the audio output unit of the mobile terminal and write it into the cache; an audio acquisition processing module, used to obtain the audio data collected by the audio acquisition unit of the mobile terminal and the original audio data of the audio data unit in the process of the audio output unit of the mobile terminal outputting data; a data delay processing module, used to delay the original audio data according to the audio data, and determine the corresponding reference data; a data shortening processing module, used to determine the transfer parameters corresponding to the audio data, and perform transfer parameter shortening processing on the audio data according to the transfer parameters to obtain processed data; a data echo cancellation module, used to perform echo cancellation on the processed data according to the reference data to obtain the echo-cancelled data after cancellation; an output data generation module, used to perform post-filtering processing on the echo-cancelled data to obtain the corresponding output data.

[0015] In order to solve the above problems, an embodiment of the present application discloses an electronic device, including: a processor; and a memory, on which executable code is stored. When the executable code is executed, the processor executes one or more methods described in the above embodiments.

[0016] In order to solve the above problems, the embodiments of the present application disclose one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes one or more methods described in the above embodiments.

[0017] Compared with the prior art, the embodiments of the present application have the following advantages:

[0018] In an embodiment of the present application, the original audio data corresponding to the data output by the audio output unit can be delayed according to the audio data collected by the audio collection unit to determine the reference data corresponding to the original audio data; and the transfer parameters corresponding to the audio data are determined, and then the audio data is subjected to transfer parameter shortening according to the transfer parameters, and the reverberation data in the audio data is filtered to obtain the processed data. After the reference data and the processed data are determined, the processed data can be echo-cancelled according to the reference data to eliminate the echo data in the processed data and determine the echo-cancelled data. Then, the echo-cancelled data is post-filtered to filter the residual noise and obtain the corresponding output data. In an embodiment of the present application, by performing transfer parameter shortening on the audio data to filter the reverberation data, and then performing echo cancellation, the processes of filtering the reverberation data and echo cancellation can be separated, thereby reducing the influence of the reverberation data on the echo cancellation process and reducing the computational complexity of the echo cancellation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of an echo cancellation method according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of an echo cancellation method according to an embodiment of the present application;

[0021] Figure 3A It is a flowchart of data delay in one embodiment of the present application;

[0022] Figure 3B is a schematic diagram of the echo cancellation process of an embodiment of the present application;

[0023] Figure 4 It is a flowchart of a voice wake-up method according to an embodiment of the present application;

[0024] Figure 5A is a structural diagram of a speech recognition system according to an embodiment of the present application;

[0025] Figure 5B It is a flowchart of a speech recognition method according to an embodiment of the present application;

[0026] Figure 6 is a flowchart of an echo cancellation method according to another embodiment of the present application;

[0027] Figure 7 is a schematic structural diagram of an echo cancellation device according to an embodiment of the present application;

[0028] Figure 8 This is a schematic diagram of the structure of a voice wake-up device according to an embodiment of the present application;

[0029] Fig. 9 is a structural diagram of a speech recognition device according to an embodiment of the present application;

[0030] Fig.10 is a structural schematic diagram of an echo cancellation device according to another embodiment of the present application;

[0031] Fig.11 It is a schematic diagram of the structure of a device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0033] The embodiments of the present application can be applied in the field of voice recognition, which is an interdisciplinary subject. Its purpose is to complete the voice communication between the user and the machine so that the machine can understand what the user is saying. The fields involved in voice recognition technology include: signal processing, pattern recognition, probability theory and information theory, vocalization mechanism and auditory mechanism, artificial intelligence, etc.

[0034] In the field of speech recognition, the collected speech audio may contain data such as the user's audio and the audio played by the device. Before the speech audio is recognized, it is usually processed by acoustic echo cancellation (AEC) to eliminate the echo in the speech audio and obtain more accurate audio recognition results. Among them, echo cancellation technology uses the echo cancellation method, that is, by estimating the size of the echo signal, and then subtracting this estimated value from the received signal to cancel the echo.

[0035] The embodiments of the present application can be applied to various mobile devices including audio input / output units, such as mobile terminals (such as mobile phones), vehicle-mounted devices and other mobile terminals. Among them, the audio input unit may include a microphone, etc., and the audio output unit may include a speaker, etc. For example, it can be applied to the scenario of performing echo cancellation processing on the device echo of the mobile terminal for subsequent voice recognition, wherein the device echo refers to the audio played by the device that is collected and recorded by the device again after being transmitted through the air, and the device echo of the mobile terminal can be understood as the audio echo played by the speaker of the mobile terminal. For example, the embodiments of the present application can be applied to navigation scenarios, music playback scenarios, TV playback scenarios, etc. of mobile terminals (such as mobile phones), such as Figure 1As shown, taking the application waking up the mobile terminal during navigation as an example, the user sends a request to the cloud server, and the cloud server returns the corresponding data to the mobile terminal to perform navigation service. In the navigation service, when the user sends voice audio to wake up the target device, the audio output unit of the mobile terminal (such as a speaker) may be playing a navigation prompt sound. The audio data collected by the audio acquisition unit (such as a microphone) of the mobile terminal includes the user's voice audio and the audio (navigation prompt sound) output by the audio output unit. The method of the embodiment of the present application can filter the audio of the navigation prompt sound to obtain clearer audio, interrupt the playback of the mobile terminal speaker based on the audio after echo elimination, and wake up the mobile terminal.

[0036] The following describes the embodiments of the present application by taking the application of the embodiments of the present application on a mobile terminal as an example.

[0037] Specifically, Figure 1 As shown, the mobile terminal of the embodiment of the present application obtains the original audio data to be played from the cloud server, writes it into the cache, and waits for playback. During the playback process, the audio data collected by the mobile terminal microphone may contain the sound (echo) played by the speaker of the mobile terminal. The embodiment of the present application can obtain the original audio data corresponding to the collected audio data and the data played by the device, and delay the original audio data to determine the corresponding reference data; determine the echo path in the audio data to determine the transfer parameters corresponding to the audio data. Then, according to the transfer parameters, the audio data is shortened (shortening) to filter the reverberation data in the audio data to obtain the processed data. After determining the reference data and the processed data, the processed data is echo-cancelled (AEC) according to the reference data. And the echo-cancelled data after echo cancellation is post-filtered to obtain the corresponding output data.

[0038] Among them, the delay of data played by devices such as speakers is usually caused by hardware reasons. Therefore, the echo cancellation method of devices such as speakers usually adopts a hard reference method, specifically, the original data of the played data is collected by hardware circuits and delayed for filtering. The delay of data played by the mobile terminal is more complicated, and the delay caused by data playback is usually uncertain. Therefore, the echo cancellation method of the mobile terminal usually adopts a soft reference method for echo cancellation. Specifically, the reasons for the delay of data played by the mobile terminal mainly include delay drift and delay jitter. Delay drift refers to the time difference between the original audio data and the collected audio data due to the inconsistency between the clocks of the mobile terminal's speaker and microphone; delay jitter refers to the fact that after the mobile terminal receives the data to be output, it will write the data to be output into the cache, and the application's playback thread will obtain and play the data to be output. However, the time required for each playback thread to obtain and play the data is different, resulting in a time difference between the original audio data and the collected audio data. In an embodiment of the present application, the original audio data can be delayed by combining a fixed delay and a dynamic delay to align the original audio data with a time delay with the collected audio data. For example, the delay of sound reception of different mobile devices is different. Specifically, a fixed delay duration can be pre-set according to the device situation, and the original audio data can be processed with a fixed delay according to the fixed delay duration. For example, a fixed delay duration of 100ms or 200ms can be adopted. After the original audio data is fixedly delayed, the audio data and the original audio data after the fixed delay can be weighted analyzed to determine the dynamic delay duration corresponding to the original audio data after the fixed delay, and according to the dynamic delay duration, the original audio data after the fixed delay can be dynamically delayed to align the delayed original audio data with the audio data. Among them, the process of weighted analysis of the audio data and the original audio data after the fixed delay can be analyzed using the generalized cross correlation phase transformation method (Generalized Cross Correlation PHAse Transformation, GCC-PHAT) to determine the corresponding dynamic delay duration.

[0039] After aligning the audio data with the reference data, the propagation path information corresponding to the audio data can be determined, and the corresponding transfer parameters can be determined based on the propagation path information, and then the audio data can be shortened based on the transfer parameters to filter the reverberation data in the audio data to obtain processed data. The transfer parameters can be understood as impulse response (IR), which refers to a description method used in acoustics to reflect the reverberation characteristics of an object. The corresponding reverberation data can be determined through the transfer parameters.

[0040] After filtering the reverberation data in the audio data, echo cancellation can be performed through an adaptive filter (AdaptiveFilter), wherein the adaptive filter is a device that automatically adjusts the filter coefficients based on the estimation of the statistical characteristics of the input and output signals and adopts a specific algorithm to achieve better filtering characteristics. Specifically, the reference data and the processed data can be used as the input data of the adaptive filter, and the elimination parameters (or coefficients) of the adaptive filter can be determined in combination with the output data of the adaptive filter. According to the elimination parameters, the echo data is simulated, and then the audio data is echo-cancelled according to the echo data to obtain linear output echo cancellation data and echo data. Among them, the adaptive filter can use the AEC algorithm of the Blind Source Separation (BSS) idea to perform echo cancellation.

[0041] After the processed data is subjected to echo cancellation processing, post-filtering processing can be performed through Wiener Filtering. The essence of Wiener Filtering is to minimize the mean square value of the estimation error (defined as the difference between the expected output and the actual output of the filter). Specifically, the linear echo data and the echo cancellation data can be input into the Wiener filter to determine the corresponding filtering parameters based on the echo data and the echo cancellation data determined by the adaptive filter, and then the echo cancellation data can be subjected to post-filtering processing based on the filtering parameters and the echo data.

[0042] In an embodiment of the present application, the original audio data corresponding to the data output by the audio output unit can be delayed according to the audio data collected by the audio collection unit to determine the reference data corresponding to the original audio data; and the transfer parameters corresponding to the audio data can be determined, and then the audio data can be shortened according to the transfer parameters, and the reverberation data in the audio data can be filtered to obtain the processed data. After the reference data and the processed data are determined, the processed data can be echo-cancelled according to the reference data to eliminate the echo data in the processed data and determine the echo-cancelled data. Then, the echo-cancelled data can be post-filtered to filter the residual noise and obtain the corresponding output data. In an embodiment of the present application, the processes of filtering the reverberation data and echo cancellation can be separated, which reduces the influence of the reverberation data on the echo cancellation process and reduces the computational complexity of the echo cancellation process.

[0043] Taking the application of the embodiment of the present application to a mobile terminal as an example, the present application provides an echo cancellation method that can be executed by the mobile terminal, specifically, Figure 2 As shown, the echo cancellation method includes:

[0044] Step 202, obtain the original audio data corresponding to the audio data collected by the audio acquisition unit and the data output by the audio output unit. Among them, the audio acquisition unit can also be called a microphone, and the mobile terminal can collect audio data through the audio acquisition unit. The audio output unit can also be called a speaker, a player, etc. The audio acquisition unit can be an audio output unit of the mobile terminal, or it can be an audio data unit of other devices, such as a player of a music player, a television, etc. In the scenario where the audio output unit outputs audio, the audio data may include the user's audio and the echo of the audio output by the audio output unit and the corresponding reverberation data. The embodiment of the present application can filter the echo of the audio output by the audio output unit in the audio data and the corresponding reverberation data.

[0045] After acquiring the audio data and the original audio data, the mobile terminal may perform delay processing on the original audio data according to the audio data to determine the corresponding reference data in step 204. The mobile terminal may delay the original audio data to align the original audio data with the audio data.

[0046] The original audio data can be delayed by a combination of fixed delay and dynamic delay. Optionally, as an embodiment, the original audio data is delayed based on the audio data to obtain corresponding reference data, including: obtaining the original audio data and audio data within the target time period; performing fixed delay processing on the original audio data within the target time period according to the fixed delay duration; performing dynamic delay processing on the original audio data after the fixed delay based on the audio data to obtain corresponding reference data. The mobile terminal can cache the audio data and the original audio data for a period of time, and then delay the audio data and the original audio data within the period of time. Specifically, after the mobile terminal obtains the audio data and the original audio data, it can cache the audio data and the original audio data, and in the case of caching the audio data and the original audio data of the preset duration, the original audio data within the period is fixedly delayed, and then the corresponding dynamic delay duration is determined based on the correlation information between the audio data and the original audio data after the fixed delay, and the original audio data after the fixed delay is delayed based on the dynamic delay duration to obtain reference data.

[0047] The embodiment of the present application can also divide the audio data within the target time period and the original audio data after a fixed delay into multiple groups of segmented data, and determine the dynamic delay duration corresponding to each segmented data. Specifically, in an optional embodiment, the original audio data after a fixed delay is subjected to dynamic delay processing based on the audio data, including: dividing the audio data within the target time period and the original audio data after a fixed delay to obtain segmented data; determining the processing time period corresponding to each group of segmented data; within the processing time period, performing weighted analysis on the segmented data to determine the corresponding relevance information; based on the relevance information, determining the dynamic delay duration corresponding to the segmented data; based on the dynamic delay duration, performing dynamic delay processing on the original audio data after a fixed delay within the target time period to obtain parameter data.

[0048] The audio data in the target time period and the original data after a fixed delay are divided to obtain multiple groups of segmented data. The mobile terminal can be divided by frame, by frequency band, or according to a preset segmentation duration, which can be set according to the needs. After the segmented data is determined, the corresponding processing time period can be assigned to the segmented data, and the segmented data can be processed during the processing period. In an optional example, multiple groups of segmented data in the previous time period can be evenly distributed to the next time period for calculation to make more reasonable use of computing resources. During the processing period, the audio data in the segmented data and the original audio data after a fixed delay can be weighted analyzed to determine the corresponding correlation information, and according to the correlation information, the dynamic delay duration corresponding to the segmented data is determined. Then, according to the dynamic delay duration, the original audio data with a fixed delay corresponding to the dynamic delay duration in the target time period is dynamically delayed to obtain parameter data for subsequent processing. In an optional example, the weighted analysis of the audio data and the original audio data can be performed by first performing Fourier transform on the audio data and the original audio data, and performing weighted analysis on the data after the Fourier transform according to formula A to obtain the analysis result as the correlation information. After the correlation information is determined, the correlation information can be converted into a dynamic delay duration for dynamic delay processing.

[0049] Formula A:

[0050] Among them, X(t,f) and D(t,f) are the original audio data and audio data after Fourier transformation respectively.

[0051] After the original audio data is subjected to fixed delay processing and dynamic delay processing so that the reference data and the audio data are aligned. In step 206, the mobile terminal can determine the transfer parameters corresponding to the audio data, and perform transfer parameter shortening processing on the audio data according to the transfer parameters to obtain processed data. The mobile terminal can analyze the audio data, determine the corresponding propagation path, and determine the corresponding transfer parameters according to the propagation path. After determining the transfer parameters, the mobile terminal can determine the reverberation data corresponding to the user's audio and the reverberation data corresponding to the audio of the audio output unit according to the transfer parameters, and filter out the reverberation data for subsequent echo cancellation processing.

[0052] The mobile terminal can convert the audio data into spectrum data, and analyze the spectrum data to determine the corresponding parameters. Specifically, as an optional embodiment, the determination of the transfer parameters corresponding to the audio data includes: transforming the audio data to obtain spectrum data; determining the corresponding propagation path information based on the spectrum data; and determining the corresponding transfer parameters based on the propagation path information. Among them, the frequency domain is a coordinate system used to describe the frequency characteristics of a signal. In one example, the audio data can be transformed into the frequency domain using Fourier transform to obtain spectrum data.

[0053] Specifically, the mobile terminal can transform the audio data into spectrum data, obtain the historical spectrum data corresponding to the spectrum data, and make predictions based on the historical spectrum data to determine the propagation path information. For example, the spectrum data of the previous time period of the time period corresponding to the spectrum data can be obtained and analyzed to determine the propagation path information. After the propagation path information is determined, the corresponding transmission parameters are determined based on the propagation path information, so as to perform transmission parameter shortening processing on the audio data based on the transmission parameters. Among them, the spectrum data transformed from the audio data can be used as the historical spectrum data of the subsequent spectrum data.

[0054] After the transfer parameters are determined, corresponding reverberation data can be determined based on the transfer parameters to offset the reverberation data in the audio data. Specifically, as an optional embodiment, performing transfer parameter shortening processing on the audio data based on the transfer parameters to obtain processed data includes: transforming the reference data to obtain reference spectrum data; determining corresponding reverberation data based on the transfer parameters and the reference spectrum data; performing transfer parameter shortening processing on spectrum data corresponding to the audio data based on the reverberation data to obtain processed data.

[0055] The mobile terminal can convert the reference data into the frequency domain to obtain the reference spectrum data. The way of converting the reference data into the frequency domain can refer to the conversion process of the audio data. After determining the reference spectrum data, the corresponding reverberation data can be determined based on the transfer parameters, the reference spectrum data and the spectrum data. Among them, the reverberation data can include the reverberation data corresponding to the user's audio data and the reverberation data corresponding to the data output by the audio data unit. Then, based on the reverberation data, the spectrum data corresponding to the audio data is shortened by the transfer parameters to filter the reverberation data in the spectrum data corresponding to the audio data to obtain the processed data.

[0056] After determining the reference data and the processed data, the mobile terminal may perform echo cancellation on the processed data according to the reference data in step 208 to obtain echo-cancelled data after cancellation. After determining the reference data and the processed data, the mobile terminal may use the reference data and the processed data as input data of the adaptive filter, and determine the coefficients (or parameters) of the adaptive filter in combination with the corresponding output data, so as to determine the echo data according to the coefficients of the adaptive filter and the reference data, and perform echo cancellation on the processed data according to the echo data to obtain the echo-cancelled data.

[0057] The mobile terminal can determine the corresponding elimination parameters (or elimination coefficients, coefficients of adaptive filters, etc.) according to the echo cancellation data of the historical processing data corresponding to the processing data to perform echo cancellation. Specifically, as an optional embodiment, the echo cancellation of the processing data according to the reference data to obtain the echo cancellation data after cancellation includes: determining the corresponding echo data according to the elimination parameters corresponding to the processing data and the reference spectrum data, the elimination parameters are determined according to the echo cancellation data of the historical processing data corresponding to the processing data; performing echo cancellation on the processing data according to the echo data to obtain the echo cancellation data. In an optional embodiment, after determining the echo cancellation data, the mobile terminal can adjust the corresponding elimination parameters according to the echo cancellation data of the processing data. The mobile terminal can obtain the processing data of the time period before the corresponding time period of the processing data as the historical processing data. For example, the mobile terminal can obtain the processing data of the previous time period of the corresponding time period of the processing data as the historical processing data. Then, the elimination parameters of the historical processing data are obtained, and the elimination parameters of the historical processing data can be adjusted according to the echo cancellation data corresponding to the historical processing data to improve the accuracy of the elimination parameters. After the cancellation parameters are determined, the corresponding echo data are simulated according to the cancellation parameters and the reference spectrum data, and the echo in the processing data is offset according to the echo data to perform echo cancellation to obtain echo cancellation data. After the echo cancellation data is determined, the cancellation parameters can be further adjusted according to the echo cancellation data to perform the next echo cancellation process more accurately.

[0058] After determining the echo cancellation data, the mobile terminal may perform post-filtering processing on the echo cancellation data to obtain corresponding output data in step 210. The mobile terminal may input the echo cancellation data and the echo data into a Wiener filter to further perform post-filtering processing on the echo cancellation data, eliminate the residual echo in the echo cancellation data, and obtain corresponding output data.

[0059] The Wiener filter can determine corresponding parameters based on the echo cancellation data and the echo data to simulate noise and perform further filtering. Specifically, as an optional embodiment, the post-filtering of the echo cancellation data to obtain the corresponding output data includes: determining filtering parameters based on the echo data and the echo cancellation data; post-filtering the echo cancellation data based on the filtering parameters and the echo data; converting the post-filtered data to determine the corresponding output audio data as the output data. The Wiener filter determines corresponding filtering parameters based on the echo cancellation data and the echo data, simulates noise data based on the filtering parameters and the echo data, and cancels the noise in the echo cancellation data based on the noise data to obtain the post-filtered data, and inversely converts the post-filtered data to convert the data in the frequency domain into output audio data to output as the output data.

[0060] After determining the output data, the mobile terminal can identify the output data and determine the corresponding recognition result. The mobile terminal can perform voice recognition on the output data to determine the text information. And perform semantic recognition on the text information to determine the key information, and perform corresponding processing based on the key information. In an optional embodiment, the mobile terminal can perform voice recognition on the output data based on automatic speech recognition technology (Automatic Speech Recognition, ASR) to obtain text information. And perform semantic recognition on the text information based on natural language processing (Natural Language Processing, NLP). Among them, ASR is a technology that converts human speech into text. NLP is an intersection of computer science, artificial intelligence and linguistics, and its purpose is to enable computers to process or "understand" natural language.

[0061] The mobile terminal can wake up the mobile terminal based on the key information, for example, it can wake up the navigation phone. The mobile terminal can also control the target object based on the key information, for example, it can control the navigation destination in the navigation application in the mobile terminal, the music and video in the playback application, etc. The mobile terminal can also determine the corresponding target object based on the key information and control the target object through the mobile terminal. For example, the mobile terminal can control the TV, fan, air conditioner and other equipment.

[0062] In an embodiment of the present application, the original audio data corresponding to the data output by the audio output unit can be delayed according to the audio data collected by the audio collection unit to determine the reference data corresponding to the original audio data; and the transfer parameters corresponding to the audio data can be determined, and then the audio data can be shortened according to the transfer parameters, and the reverberation data in the audio data can be filtered to obtain the processed data. After the reference data and the processed data are determined, the processed data can be echo-cancelled according to the reference data to eliminate the echo data in the processed data and determine the echo-cancelled data. Then, the echo-cancelled data can be post-filtered to filter the residual noise and obtain the corresponding output data. In an embodiment of the present application, the processes of filtering the reverberation data and echo cancellation can be separated, which reduces the influence of the reverberation data on the echo cancellation process and reduces the computational complexity of the echo cancellation process.

[0063] Based on the above embodiments, the embodiments of the present application also disclose an echo elimination method that can be executed by a mobile terminal. The method includes: a process of performing fixed delay and dynamic delay on the original audio data to align the original audio data and the audio data, and a process of eliminating the aligned data.

[0064] Among them, Figure 3A As shown in FIG. 1 , the process of performing fixed delay and dynamic delay on the original audio data includes:

[0065] Step 302: Acquire original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit.

[0066] Step 304: Acquire original audio data and audio data within the target time period.

[0067] Step 306: Perform fixed delay processing on the original audio data within the target time period according to the fixed delay duration.

[0068] Step 308: segment the audio data in the target time period and the original audio data after the fixed delay to obtain segmented data.

[0069] Step 310: Determine the processing time period corresponding to each group of segmented data.

[0070] Step 312: During the processing period, weighted analysis is performed on the segmented data to determine corresponding relevance information.

[0071] Step 314: Determine the dynamic delay duration corresponding to the segmented data according to the correlation information.

[0072] Step 316: Perform dynamic delay processing on the original audio data after a fixed delay within the target time period according to the dynamic delay duration to obtain reference data.

[0073] In an embodiment of the present application, the original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit can be cached, and when the audio data and the original audio data cache reach the preset duration corresponding to the target time period, the audio data and the original audio data in the target time period are processed. Specifically, the mobile terminal can delay the original audio data in the target time period according to the fixed delay duration, and divide the audio data in the target time period and the original audio data after the fixed delay to obtain segmented data, and determine the processing time period corresponding to the segmented data, wherein the processing time period can be determined by averaging the next time period of the target time period. After determining the processing time period corresponding to the segmented data, the segmented data is weighted analyzed in the processing time period to determine the correlation degree information between the audio data in the segmented data and the original audio data. And based on the correlation degree information, the dynamic delay duration corresponding to the segmented data is determined, so that the original audio data after the fixed delay in the target time period is dynamically delayed according to the dynamic delay duration.

[0074] An existing method of aligning audio data with original audio data is to align the original audio data with the audio data by matching the spectrum corresponding to the audio data and the original audio data. However, this method can usually only obtain a frame-level (generally 4ms) delay. Compared with the method of aligning the original audio data with the audio data by spectrum matching, the method of the embodiment of the present application can align the data by combining a fixed delay with a dynamic delay, which can align the data more accurately. In addition, the weighted analysis process of the original audio data and the audio data can be evenly distributed to a time period for processing, making more reasonable use of computing resources.

[0075] like Figure 3B As shown, the process of eliminating the aligned data includes:

[0076] Step 318: transform the audio data to obtain spectrum data, and transform the reference data to obtain reference spectrum data.

[0077] Step 320: Determine corresponding propagation path information based on the spectrum data.

[0078] Step 322: Determine corresponding transmission parameters based on the propagation path information.

[0079] Step 324: Determine corresponding reverberation data according to the transferred parameters and the reference spectrum data.

[0080] Step 326: Based on the reverberation data, the spectrum data corresponding to the audio data is subjected to a transfer parameter shortening process to obtain processed data.

[0081] Step 328: Determine the corresponding echo data according to the cancellation parameters corresponding to the processed data and the reference spectrum data, wherein the cancellation parameters are determined according to the echo cancellation data of the historical processed data corresponding to the processed data.

[0082] Step 330: According to the echo data, echo cancellation is performed on the processed data to obtain echo cancellation data. According to the echo cancellation data of the processed data, corresponding cancellation parameters are adjusted.

[0083] Step 332: Determine filtering parameters based on the echo data and the echo cancellation data.

[0084] Step 334: perform post-filtering processing on the echo cancellation data according to the filter parameters and the echo data.

[0085] Step 336: Convert the post-filtered data to determine the corresponding output audio data as output data.

[0086] Step 338: Identify the output data and determine the corresponding identification result.

[0087] In an embodiment of the present application, the audio data and the reference data can be transformed into the frequency domain to obtain the spectrum data and the reference spectrum data. Then, the corresponding transfer parameters are determined based on the spectrum data, and the corresponding reverberation data is determined based on the transfer parameters and the reference spectrum data to offset the reverberation in the spectrum data corresponding to the audio data to obtain the processed data. After that, the processed data and the reference spectrum data can be input into the adaptive filter to determine the corresponding elimination parameters, so as to determine the echo data based on the elimination parameters and the reference spectrum data, and offset the echo in the processed data based on the echo data to obtain the echo cancellation data. After that, the echo cancellation data and the echo data are input into the Wiener filter to perform post-filtering processing on the echo cancellation data, and the post-filtered data are converted to determine the corresponding output audio data as the output data. After that, the output data can be identified to perform corresponding processing.

[0088] The embodiments of the present application can also be applied to wake-up scenarios during audio playback on a mobile terminal. For example, the embodiments of the present application can be applied to echo cancellation scenarios of audio data during navigation on a mobile terminal to eliminate navigation prompt audio data contained in the audio data, so as to more accurately wake up the mobile terminal. Specifically, based on the above embodiments, the embodiments of the present application also disclose a voice wake-up method that can be executed by a mobile terminal, such as Figure 4 As shown, the method includes:

[0089] Step 402: Acquire original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit.

[0090] Step 404: Delay processing is performed on the original audio data according to the audio data to determine corresponding reference data.

[0091] Step 406: Determine the transfer parameters corresponding to the audio data, and perform transfer parameter shortening processing on the audio data according to the transfer parameters to obtain processed data.

[0092] Step 408: Perform echo cancellation on the processed data according to the reference data to obtain echo cancelled data.

[0093] Step 410: perform post-filtering processing on the echo cancellation data to obtain corresponding output audio data.

[0094] Step 412: wake up the corresponding target object according to the output audio data.

[0095] The processing process of the voice wake-up method of the embodiment of the present application is similar to or the same as the processing process of the above-mentioned echo cancellation method. For details, please refer to the process of the method in the above-mentioned embodiment, which will not be repeated here.

[0096] In an embodiment of the present application, the original audio data corresponding to the data output by the audio output unit can be delayed according to the audio data collected by the audio acquisition unit to determine the reference data corresponding to the original audio data; and the transmission parameters corresponding to the audio data are determined, and then the transmission parameters are shortened according to the transmission parameters, and the reverberation data in the audio data is filtered to obtain the processed data. After the reference data and the processed data are determined, the processed data can be echo-cancelled according to the reference data to eliminate the echo data in the processed data and determine the echo-cancelled data. Then the echo-cancelled data is post-filtered to filter the residual noise to obtain the corresponding output audio data. Then the output audio can be identified to determine the corresponding text information, and the text information is identified to determine the key information corresponding to the target object. In the case where there is a wake-up keyword in the key information, the corresponding target object (such as a mobile terminal) is awakened. In an embodiment of the present application, the process of filtering the reverberation data and the echo cancellation can be separated, which reduces the influence of the reverberation data on the echo cancellation process and reduces the computational complexity of the echo cancellation process. The output audio data obtained after echo cancellation in the above manner can more accurately restore the user's audio and improve the wake-up success rate of the mobile terminal.

[0097] The embodiments of the present application can be applied in the scenario of voice recognition of a mobile terminal to control a target object, and the target object can be an electronic device connected to the mobile terminal. Specifically, the mobile terminal can perform echo cancellation processing on the audio data and perform corresponding recognition on the echo-cancelled audio data to determine the corresponding instruction, which can be used to control the target object. For example, in Figure 5A In the example shown, the target object may be an air conditioner and a sweeping robot connected to a mobile terminal. The user may issue a voice command to the mobile terminal to control the air conditioner and the sweeping robot through the mobile terminal. Specifically, based on the above embodiment, the present application embodiment further discloses a voice recognition method that can be executed by a mobile terminal, such as Figure 5B As shown, the method includes:

[0098] Step 502: Acquire original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit.

[0099] Step 504: Delay processing is performed on the original audio data according to the audio data to determine corresponding reference data.

[0100] Step 506: Determine the transfer parameters corresponding to the audio data, and perform transfer parameter shortening processing on the audio data according to the transfer parameters to obtain processed data.

[0101] Step 508: Perform echo cancellation on the processed data according to the reference data to obtain echo cancelled data.

[0102] Step 510: perform post-filtering processing on the echo cancellation data to obtain corresponding output audio data.

[0103] Step 512: Perform speech recognition on the audio data to determine text information.

[0104] Step 514: perform semantic recognition on the text information to determine key information.

[0105] Step 516: Determine the corresponding target object based on the key information, and perform corresponding processing on the target object.

[0106] The processing process of the speech recognition method of the embodiment of the present application is similar to or the same as the processing process of the above-mentioned echo cancellation method. For details, please refer to the process of the method of the above-mentioned embodiment, which will not be repeated here.

[0107] In the embodiment of the present application, the original audio data corresponding to the data output by the audio output unit can be delayed according to the audio data collected by the audio collection unit to determine the reference data corresponding to the original audio data; and the transmission parameters corresponding to the audio data are determined, and then the transmission parameters are shortened according to the transmission parameters, and the reverberation data in the audio data is filtered to obtain the processed data. After the reference data and the processed data are determined, the processed data can be echo-cancelled according to the reference data to eliminate the echo data in the processed data and determine the echo-cancelled data. Then the echo-cancelled data is post-filtered to filter the residual noise to obtain the corresponding output audio data. Then the output audio can be speech recognized to determine the corresponding text information, and the text information is semantically recognized to determine the key information corresponding to the target object. And according to the key information, the target object is processed accordingly. In the embodiment of the present application, the process of filtering the reverberation data and the echo cancellation can be separated, which reduces the influence of the reverberation data on the echo cancellation process and reduces the computational complexity of the echo cancellation process.

[0108] On the basis of the above embodiments, the embodiment of the present application further provides an echo cancellation method, which can be applied in a mobile terminal to perform echo cancellation processing on audio data collected during the audio playback process of the mobile terminal. Specifically, Figure 6 As shown, the method includes:

[0109] Step 602: Obtain original audio data to be output by the audio output unit of the mobile terminal, and write it into the cache.

[0110] Step 604: in the process of the audio output unit of the mobile terminal outputting data, the audio data collected by the audio collection unit of the mobile terminal and the original audio data of the audio data unit are obtained.

[0111] Step 606: Delay the original audio data according to the audio data to determine corresponding reference data.

[0112] Step 608: Determine the transfer parameters corresponding to the audio data, and perform transfer parameter shortening processing on the audio data according to the transfer parameters to obtain processed data.

[0113] Step 610: Perform echo cancellation on the processed data according to the reference data to obtain echo cancelled data.

[0114] Step 612: Perform post-filtering processing on the echo cancellation data to obtain corresponding output data.

[0115] The processing process of the method in the embodiment of the present application is similar to or the same as the processing process of the above-mentioned method. For details, please refer to the process of the method in the above-mentioned embodiment, which will not be repeated here.

[0116] In the embodiment of the present application, the data to be output can be obtained and written into the cache. In the process of the audio output unit of the mobile terminal outputting data, the original audio data corresponding to the data output by the audio output unit can be delayed according to the audio data collected by the audio collection unit to determine the reference data corresponding to the original audio data; and the transfer parameter corresponding to the audio data is determined, and then the audio data is shortened according to the transfer parameter, and the reverberation data in the audio data is filtered to obtain the processed data. After the reference data and the processed data are determined, the processed data can be echo-cancelled according to the reference data to eliminate the echo data in the processed data and determine the echo-cancelled data. Then, the echo-cancelled data is post-filtered to filter the residual noise and obtain the corresponding output data. In the embodiment of the present application, by performing the transfer parameter shortening process on the audio data to filter the reverberation data, and then performing the echo cancellation, the process of filtering the reverberation data and the echo cancellation can be separated, which reduces the influence of the reverberation data on the echo cancellation process and reduces the computational complexity of the echo cancellation process.

[0117] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0118] Based on the above embodiment, this embodiment further provides an echo cancellation device. Figure 6 , specifically it can include the following modules:

[0119] The audio data acquisition module 702 is used to acquire the original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit.

[0120] The reference data acquisition module 704 is used to perform delay processing on the original audio data according to the audio data to determine corresponding reference data.

[0121] The processed data acquisition module 706 is used to determine the transmission parameters corresponding to the audio data, and perform transmission parameter shortening processing on the audio data according to the transmission parameters to obtain processed data.

[0122] The echo cancellation data acquisition module 708 is used to perform echo cancellation on the processed data according to the reference data to obtain echo cancellation data.

[0123] The output data acquisition module 710 is used to perform post-filtering processing on the echo cancellation data to obtain corresponding output data.

[0124] In summary, in the embodiment of the present application, the original audio data corresponding to the data output by the audio output unit can be delayed according to the audio data collected by the audio acquisition unit to determine the reference data corresponding to the original audio data; and the transfer parameters corresponding to the audio data are determined, and then the audio data is subjected to transfer parameter shortening processing according to the transfer parameters, and the reverberation data in the audio data is filtered to obtain the processed data. After the reference data and the processed data are determined, the processed data can be echo-cancelled according to the reference data to eliminate the echo data in the processed data and determine the echo-cancelled data. Then, the echo-cancelled data is post-filtered to filter the residual noise and obtain the corresponding output data. In the embodiment of the present application, by performing transfer parameter shortening processing on the audio data to filter the reverberation data, and then performing echo cancellation, the processes of filtering the reverberation data and echo cancellation can be separated, thereby reducing the influence of the reverberation data on the echo cancellation process and reducing the computational complexity of the echo cancellation process.

[0125] Based on the above embodiment, this embodiment further provides an echo cancellation device, which may specifically include the following modules:

[0126] The audio acquisition module is used to obtain the original audio data corresponding to the audio data collected by the audio acquisition unit and the data output by the audio output unit.

[0127] The trigger processing module is used to obtain the original audio data and audio data within the target time period.

[0128] The fixed delay processing module is used to perform fixed delay processing on the original audio data within the target time period according to the fixed delay duration.

[0129] The segmentation processing module is used to segment the audio data within the target time period and the original audio data after a fixed delay to obtain segmented data.

[0130] The processing period determination module is used to determine the processing period corresponding to each group of segmented data.

[0131] The relevance determination module is used to perform weighted analysis on the segmented data within a processing period to determine the corresponding relevance information.

[0132] The dynamic delay duration determination module is used to determine the dynamic delay duration corresponding to the segmented data based on the correlation degree information.

[0133] The dynamic delay processing module is used to perform dynamic delay processing on the original audio data after a fixed delay in a target time period according to the dynamic delay duration to obtain parameter data.

[0134] The frequency domain conversion processing module is used to transform the audio data to obtain spectrum data, and to transform the reference data to obtain reference spectrum data.

[0135] The propagation path determination module is used to determine the corresponding propagation path information based on the spectrum data.

[0136] The transmission parameter determination module is used to determine the corresponding transmission parameters according to the propagation path information.

[0137] The reverberation data determination module is used to determine the corresponding reverberation data according to the transfer parameters and the reference spectrum data.

[0138] The shortening processing module is used to perform a transfer parameter shortening process on the spectrum data corresponding to the audio data according to the reverberation data to obtain processed data.

[0139] The echo data determination module is used to determine the corresponding echo data according to the cancellation parameters corresponding to the processed data and the reference spectrum data, wherein the cancellation parameters are determined according to the echo cancellation data of the historical processed data corresponding to the processed data.

[0140] The echo cancellation module is used to perform echo cancellation on the processed data according to the echo data to obtain echo cancellation data and adjust the corresponding cancellation parameters according to the echo cancellation data of the processed data.

[0141] The filter parameter determination module is used to determine the filter parameters according to the echo data and the echo cancellation data.

[0142] The post-filtering processing module is used to perform post-filtering processing on the echo cancellation data according to the filtering parameters and the echo data.

[0143] The inverse conversion processing module is used to convert the data after the post-filtering processing to determine the corresponding output audio data as the output data.

[0144] The recognition processing module is used to recognize the output data and determine the corresponding recognition result.

[0145] In an embodiment of the present application, the original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit can be cached, and when the audio data and the original audio data cache reach the preset duration corresponding to the target time period, the audio data and the original audio data in the target time period are processed. Specifically, the mobile terminal can delay the original audio data in the target time period according to the fixed delay duration, and divide the audio data in the target time period and the original audio data after the fixed delay to obtain segmented data, and determine the processing time period corresponding to the segmented data, wherein the processing time period can be determined by averaging the next time period of the target time period. After determining the processing time period corresponding to the segmented data, the segmented data is weighted analyzed in the processing time period to determine the correlation degree information between the audio data in the segmented data and the original audio data. And based on the correlation degree information, the dynamic delay duration corresponding to the segmented data is determined, so that the original audio data after the fixed delay in the target time period is dynamically delayed according to the dynamic delay duration. Compared with the method of aligning the original audio data and the audio data by spectrum matching, the method of the embodiment of the present application can align the data by combining fixed delay with dynamic delay, which can align the data more accurately. In addition, the weighted analysis process of the original audio data and the audio data can be evenly distributed to a time period for processing, making more reasonable use of computing resources.

[0146] In an embodiment of the present application, the audio data and the reference data can be transformed into the frequency domain to obtain the spectrum data and the reference spectrum data. Then, the corresponding transfer parameters are determined based on the spectrum data, and the corresponding reverberation data is determined based on the transfer parameters and the reference spectrum data to offset the reverberation in the spectrum data corresponding to the audio data to obtain the processed data. After that, the processed data and the reference spectrum data can be input into the adaptive filter to determine the corresponding elimination parameters, so as to determine the echo data based on the elimination parameters and the reference spectrum data, and offset the echo in the processed data based on the echo data to obtain the echo cancellation data. After that, the echo cancellation data and the echo data are input into the Wiener filter to perform post-filtering processing on the echo cancellation data, and the post-filtered data are converted to determine the corresponding output audio data as the output data. After that, the output data can be identified to perform corresponding processing.

[0147] Based on the above embodiment, this embodiment also provides a voice wake-up device, referring to Figure 7 , specifically it can include the following modules:

[0148] The audio data determination module 802 is used to obtain the original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit.

[0149] The reference data determination module 804 is used to perform delay processing on the original audio data according to the audio data to determine corresponding reference data.

[0150] The processed data determination module 806 is used to determine the transfer parameters corresponding to the audio data, and perform transfer parameter shortening processing on the audio data according to the transfer parameters to obtain processed data.

[0151] The echo cancellation data determination module 808 is used to perform echo cancellation on the processed data according to the reference data to obtain echo cancellation data after cancellation.

[0152] The output data determination module 810 is used to perform post-filtering processing on the echo cancellation data to obtain corresponding output audio data.

[0153] The target object awakening module 812 is used to wake up the corresponding target object according to the output audio data.

[0154] In summary, in the embodiment of the present application, the original audio data corresponding to the data output by the audio output unit can be delayed according to the audio data collected by the audio acquisition unit to determine the reference data corresponding to the original audio data; and the transmission parameters corresponding to the audio data are determined, and then the transmission parameters are shortened according to the transmission parameters, and the reverberation data in the audio data is filtered to obtain the processed data. After the reference data and the processed data are determined, the processed data can be echo-cancelled according to the reference data to eliminate the echo data in the processed data and determine the echo-cancelled data. Then the echo-cancelled data is post-filtered to filter the residual noise to obtain the corresponding output audio data. Then the output audio can be identified to determine the corresponding text information, and the text information is identified to determine the key information corresponding to the target object. In the case where there is a wake-up keyword in the key information, the corresponding target object (such as a mobile terminal) is awakened. In the embodiment of the present application, the process of filtering the reverberation data and the echo cancellation can be separated, which reduces the influence of the reverberation data on the echo cancellation process and reduces the computational complexity of the echo cancellation process. The output audio data obtained after echo cancellation in the above manner can more accurately restore the user's audio and improve the wake-up success rate of the mobile terminal.

[0155] Based on the above embodiment, this embodiment also provides a speech recognition device, referring to Figure 8 , specifically it can include the following modules:

[0156] The audio data acquisition module 902 is used to obtain the original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit.

[0157] The reference data obtaining module 904 is used to perform delay processing on the original audio data according to the audio data to determine corresponding reference data.

[0158] The processed data acquisition module 906 is used to determine the transmission parameters corresponding to the audio data, and perform transmission parameter shortening processing on the audio data according to the transmission parameters to obtain processed data.

[0159] The echo cancellation data obtaining module 908 is used to perform echo cancellation on the processed data according to the reference data to obtain echo cancellation data.

[0160] The output data acquisition module 910 is used to perform post-filtering processing on the echo cancellation data to obtain corresponding output audio data.

[0161] The speech recognition module 912 is used to perform speech recognition on the audio data to determine text information.

[0162] The semantic recognition module 914 is used to perform semantic recognition on the text information to determine key information.

[0163] The target object processing module 916 is used to determine the corresponding target object according to the key information and perform corresponding processing on the target object.

[0164] In summary, in the embodiment of the present application, the original audio data corresponding to the data output by the audio output unit can be delayed according to the audio data collected by the audio acquisition unit to determine the reference data corresponding to the original audio data; and the transmission parameters corresponding to the audio data are determined, and then the transmission parameters are shortened according to the transmission parameters, and the reverberation data in the audio data is filtered to obtain the processed data. After the reference data and the processed data are determined, the processed data can be echo-cancelled according to the reference data to eliminate the echo data in the processed data and determine the echo-cancelled data. Then the echo-cancelled data is post-filtered to filter the residual noise to obtain the corresponding output audio data. Then the output audio can be speech recognized to determine the corresponding text information, and the text information is semantically recognized to determine the key information corresponding to the target object. And according to the key information, the target object is processed accordingly. In the embodiment of the present application, the process of filtering the reverberation data and the echo cancellation can be separated, which reduces the influence of the reverberation data on the echo cancellation process and reduces the computational complexity of the echo cancellation process.

[0165] Based on the above embodiment, this embodiment further provides an echo cancellation device. Fig.10 , specifically it can include the following modules:

[0166] The original data cache module 1002 is used to obtain the original audio data to be output by the audio output unit of the mobile terminal and write it into the cache.

[0167] The audio collection processing module 1004 is used to obtain the audio data collected by the audio collection unit of the mobile terminal and the original audio data of the audio data unit during the process of the audio output unit of the mobile terminal outputting data.

[0168] The data delay processing module 1006 is used to perform delay processing on the original audio data according to the audio data to determine corresponding reference data.

[0169] The data shortening processing module 1008 is used to determine the transmission parameters corresponding to the audio data, and perform transmission parameter shortening processing on the audio data according to the transmission parameters to obtain processed data.

[0170] The data echo cancellation module 1010 is used to perform echo cancellation on the processed data according to the reference data to obtain echo-cancelled data.

[0171] The output data generating module 1012 is used to perform post-filtering processing on the echo cancellation data to obtain corresponding output data.

[0172] In the embodiment of the present application, the data to be output can be obtained and written into the cache. In the process of the audio output unit of the mobile terminal outputting data, the original audio data corresponding to the data output by the audio output unit can be delayed according to the audio data collected by the audio collection unit to determine the reference data corresponding to the original audio data; and the transfer parameter corresponding to the audio data is determined, and then the audio data is shortened according to the transfer parameter, and the reverberation data in the audio data is filtered to obtain the processed data. After the reference data and the processed data are determined, the processed data can be echo-cancelled according to the reference data to eliminate the echo data in the processed data and determine the echo-cancelled data. Then, the echo-cancelled data is post-filtered to filter the residual noise and obtain the corresponding output data. In the embodiment of the present application, by performing the transfer parameter shortening process on the audio data to filter the reverberation data, and then performing the echo cancellation, the process of filtering the reverberation data and the echo cancellation can be separated, which reduces the influence of the reverberation data on the echo cancellation process and reduces the computational complexity of the echo cancellation process.

[0173] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0174] The present application embodiment provides one or more machine-readable media, on which instructions are stored, and when executed by one or more processors, an electronic device executes one or more methods as described in the above embodiments. In the present application embodiment, the electronic device includes a server, a terminal device, and the like.

[0175] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as a server (cluster), a terminal, etc. Fig.11 An exemplary apparatus 1100 that can be used to implement various embodiments described in this application is schematically shown.

[0176] For one embodiment, Fig.11 An exemplary apparatus 1100 is shown having one or more processors 1102, a control module (chip set) 1104 coupled to at least one of the (one or more) processors 1102, a memory 1106 coupled to the control module 1104, a non-volatile memory (NVM) / storage device 1108 coupled to the control module 1104, one or more input / output devices 1110 coupled to the control module 1104, and a network interface 1112 coupled to the control module 1104.

[0177] The processor 1102 may include one or more single-core or multi-core processors, and the processor 1102 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 1100 can be used as a server, terminal, or other device described in the embodiments of the present application.

[0178] In some embodiments, the apparatus 1100 may include one or more computer-readable media (e.g., memory 1106 or NVM / storage device 1108) having instructions 1114 and one or more processors 1102 combined with the one or more computer-readable media and configured to execute the instructions 1114 to implement a module to perform the actions described in the present disclosure.

[0179] For one embodiment, the control module 1104 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1102 and / or any suitable device or component in communication with the control module 1104 .

[0180] The control module 1104 may include a memory controller module to provide an interface to the memory 1106. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0181] The memory 1106 may be used, for example, to load and store data and / or instructions 1114 for the device 1100. For one embodiment, the memory 1106 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 1106 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0182] For one embodiment, control module 1104 may include one or more input / output controllers to provide an interface to NVM / storage device 1108 and input / output device(s) 1110 .

[0183] For example, NVM / storage 1108 may be used to store data and / or instructions 1114. NVM / storage 1108 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0184] NVM / storage device 1108 may include storage resources that are part of the device on which apparatus 1100 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 1108 may be accessed via (one or more) input / output devices 1110 over a network.

[0185] (One or more) input / output devices 1110 may provide an interface for the apparatus 1100 to communicate with any other appropriate device, and the input / output device 1110 may include a communication component, an audio component, a sensor component, etc. The network interface 1112 may provide an interface for the apparatus 1100 to communicate through one or more networks, and the apparatus 1100 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0186] For one embodiment, at least one of the processor(s) 1102 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 1104. For one embodiment, at least one of the processor(s) 1102 may be packaged together with the logic of one or more controllers of the control module 1104 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1102 may be integrated on the same die with the logic of one or more controllers of the control module 1104. For one embodiment, at least one of the processor(s) 1102 may be integrated on the same die with the logic of one or more controllers of the control module 1104 to form a system-on-chip (SoC).

[0187] In various embodiments, the device 1100 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 1100 may have more or fewer components and / or a different architecture. For example, in some embodiments, the device 1100 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0188] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in a memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0189] An embodiment of the present application further provides an electronic device, comprising: a processor; and a memory, on which executable code is stored, and when the executable code is executed, the processor executes one or more methods described in the embodiments of the present application.

[0190] The embodiments of the present application also provide one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes one or more methods described in the embodiments of the present application.

[0191] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0192] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0193] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0194] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0195] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0196] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0197] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0198] The above is a detailed introduction to an echo cancellation method, a voice wake-up method, a voice recognition method, an echo cancellation device, a voice wake-up device, a voice recognition device, an electronic device and a storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for a general technician in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An echo cancellation method, characterized in that: The method includes: Acquire original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit; According to the audio data, delay processing is performed on the original audio data to determine corresponding reference data; Determining a transfer parameter corresponding to the audio data; Transforming the reference data to obtain reference spectrum data; Determining corresponding reverberation data according to the transfer parameter and the reference spectrum data; According to the reverberation data, the spectrum data corresponding to the audio data is subjected to a transfer parameter shortening process to obtain processed data; performing echo cancellation on the processed data according to the reference data to obtain echo-cancelled data; Post-filtering is performed on the echo cancellation data to obtain corresponding output data.

2. The method according to claim 1, characterized in that The delay processing is performed on the original audio data according to the audio data to obtain corresponding reference data, including: Obtaining original audio data and audio data within a target period; Performing fixed delay processing on the original audio data within the target time period according to the fixed delay duration; According to the audio data, dynamic delay processing is performed on the original audio data after fixed delay to obtain corresponding reference data.

3. The method according to claim 2, characterized in that The step of performing dynamic delay processing on the original audio data after the fixed delay according to the audio data comprises: The audio data in the target time period and the original audio data after a fixed delay are segmented to obtain segmented data; Determine the processing period corresponding to each group of segmented data; During the processing period, weighted analysis is performed on the segmented data to determine corresponding relevance information; Determine the dynamic delay duration corresponding to the segmented data according to the correlation information; According to the dynamic delay duration, dynamic delay processing is performed on the original audio data that has undergone a fixed delay within a target time period to obtain parameter data.

4. The method according to claim 1, characterized in that: The determining of the transfer parameters corresponding to the audio data includes: Transforming the audio data to obtain spectrum data; Determining corresponding propagation path information according to the spectrum data; According to the propagation path information, corresponding transmission parameters are determined.

5. The method according to claim 1, characterized in that The echo cancellation of the processed data according to the reference data to obtain echo-cancelled data includes: Determining corresponding echo data according to cancellation parameters corresponding to the processed data and reference spectrum data, wherein the cancellation parameters are determined according to echo cancellation data of historical processed data corresponding to the processed data; According to the echo data, echo cancellation is performed on the processed data to obtain echo cancelled data.

6. The method according to claim 5, characterized in that Also includes: According to the echo cancellation data of the processed data, the corresponding cancellation parameters are adjusted.

7. The method according to claim 5, characterized in that The post-filtering of the echo cancellation data to obtain corresponding output data includes: Determining filtering parameters according to the echo data and the echo cancellation data; performing post-filtering processing on the echo cancellation data according to the filtering parameters and the echo data; The post-filtered data is converted to determine the corresponding output audio data as output data.

8. The method according to claim 1, characterized in that Also includes: The output data is identified to determine a corresponding identification result.

9. A voice wake-up method, characterized in that: include: Acquire original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit; According to the audio data, delay processing is performed on the original audio data to determine corresponding reference data; Determining a transfer parameter corresponding to the audio data; Transforming the reference data to obtain reference spectrum data; Determining corresponding reverberation data according to the transfer parameter and the reference spectrum data; According to the reverberation data, the spectrum data corresponding to the audio data is subjected to a transfer parameter shortening process to obtain processed data; performing echo cancellation on the processed data according to the reference data to obtain echo-cancelled data; Performing post-filtering processing on the echo cancellation data to obtain corresponding output audio data; The corresponding target object is awakened according to the output audio data.

10. A speech recognition method, characterized in that: include: Acquire original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit; According to the audio data, delay processing is performed on the original audio data to determine corresponding reference data; Determining a transfer parameter corresponding to the audio data; Transforming the reference data to obtain reference spectrum data; Determining corresponding reverberation data according to the transfer parameter and the reference spectrum data; According to the reverberation data, the spectrum data corresponding to the audio data is subjected to a transfer parameter shortening process to obtain processed data; performing echo cancellation on the processed data according to the reference data to obtain echo-cancelled data; Performing post-filtering processing on the echo cancellation data to obtain corresponding output audio data; Performing speech recognition on the audio data to determine text information; Performing semantic recognition on the text information to determine key information; According to the key information, the corresponding target object is determined, and the target object is processed accordingly.

11. An echo cancellation method, characterized in that: Applied to a mobile terminal, the method includes: Obtaining the original audio data to be output by the audio output unit of the mobile terminal and writing it into the cache; In the process of the audio output unit of the mobile terminal outputting data, obtaining the audio data collected by the audio collection unit of the mobile terminal and the original audio data of the audio data unit; According to the audio data, delay processing is performed on the original audio data to determine corresponding reference data; Determining a transfer parameter corresponding to the audio data; Transforming the reference data to obtain reference spectrum data; Determining corresponding reverberation data according to the transfer parameter and the reference spectrum data; According to the reverberation data, the spectrum data corresponding to the audio data is subjected to a transfer parameter shortening process to obtain processed data; performing echo cancellation on the processed data according to the reference data to obtain echo-cancelled data; Post-filtering is performed on the echo cancellation data to obtain corresponding output data.

12. An echo cancellation device, characterized in that: include: An audio data acquisition module, used to acquire original audio data corresponding to the audio data collected by the audio acquisition unit and the data output by the audio output unit; A reference data acquisition module, used to perform delay processing on the original audio data according to the audio data to determine corresponding reference data; A processing data acquisition module is used to determine the transfer parameters corresponding to the audio data; transform the reference data to obtain reference spectrum data; determine the corresponding reverberation data based on the transfer parameters and the reference spectrum data; perform transfer parameter shortening processing on the spectrum data corresponding to the audio data based on the reverberation data to obtain processed data; An echo cancellation data acquisition module, used to perform echo cancellation on the processed data according to the reference data to obtain echo cancellation data after cancellation; The output data acquisition module is used to perform post-filtering processing on the echo cancellation data to obtain corresponding output data.

13. A voice wake-up device, characterized in that: include: An audio data determination module, used to obtain original audio data corresponding to the audio data collected by the audio collection unit and the data output by the audio output unit; A reference data determination module, configured to perform delay processing on the original audio data according to the audio data to determine corresponding reference data; A processing data determination module is used to determine the transfer parameters corresponding to the audio data; transform the reference data to obtain reference spectrum data; determine the corresponding reverberation data based on the transfer parameters and the reference spectrum data; perform transfer parameter shortening processing on the spectrum data corresponding to the audio data based on the reverberation data to obtain processed data; an echo cancellation data determination module, configured to perform echo cancellation on the processed data according to the reference data to obtain echo cancellation data after cancellation; An output data determination module, used for performing post-filtering processing on the echo cancellation data to obtain corresponding output audio data; The target object awakening module is used to wake up the corresponding target object according to the output audio data.

14. A speech recognition device, characterized in that: include: An audio data acquisition module, used to acquire original audio data corresponding to the audio data collected by the audio acquisition unit and the data output by the audio output unit; A reference data acquisition module, used to perform delay processing on the original audio data according to the audio data to determine corresponding reference data; A processing data acquisition module is used to determine the transfer parameters corresponding to the audio data; transform the reference data to obtain reference spectrum data; determine the corresponding reverberation data based on the transfer parameters and the reference spectrum data; perform transfer parameter shortening processing on the spectrum data corresponding to the audio data based on the reverberation data to obtain processed data; An echo cancellation data acquisition module, used to perform echo cancellation on the processed data according to the reference data to obtain echo cancellation data after cancellation; An output data acquisition module, used for performing post-filtering processing on the echo cancellation data to obtain corresponding output audio data; A speech recognition module, used for performing speech recognition on the audio data to determine text information; A semantic recognition module, used to perform semantic recognition on the text information to determine key information; The target object processing module is used to determine the corresponding target object according to the key information and perform corresponding processing on the target object.

15. An echo cancellation device, characterized in that: include: The original data cache module is used to obtain the original audio data to be output by the audio output unit of the mobile terminal and write it into the cache; The audio collection processing module is used to obtain the audio data collected by the audio collection unit of the mobile terminal and the original audio data of the audio data unit during the process of the audio output unit of the mobile terminal outputting data; A data delay processing module, used to perform delay processing on the original audio data according to the audio data to determine corresponding reference data; A data shortening processing module is used to determine the transfer parameters corresponding to the audio data; transform the reference data to obtain reference spectrum data; determine the corresponding reverberation data based on the transfer parameters and the reference spectrum data; perform transfer parameter shortening processing on the spectrum data corresponding to the audio data based on the reverberation data to obtain processed data; A data echo cancellation module, configured to perform echo cancellation on the processed data according to the reference data to obtain echo-cancelled data; The output data generating module is used to perform post-filtering processing on the echo cancellation data to obtain corresponding output data.

16. An electronic device, characterized in that: include: processor; and A memory having executable codes stored thereon, which, when executed, cause the processor to perform the method according to one or more of claims 1-11.

17. One or more machine-readable media having executable codes stored thereon, which, when executed, cause a processor to perform the method of one or more of claims 1-11.

Images