Audio signal processing method, device, terminal and storage medium

Abstract

The present application relates to a method, device, terminal and storage medium for processing an audio signal, and belongs to the field of audio processing technology. The method includes: based on a beam coefficient, processing a first audio signal to obtain a second audio signal, the first audio signal is an audio signal picked up by an audio acquisition component, and the beam coefficient is used to indicate a target pickup range; based on the first audio signal and the second audio signal, determining a processing coefficient, the processing coefficient is used to enhance the audio signal within the pickup range and filter the audio signal outside the pickup range; based on the processing coefficient, processing the second audio signal to obtain a third audio signal. The present application processes the picked-up audio signal through the processing coefficient, that is, enhances the audio signal within the target pickup range, suppresses the audio signal outside the range, improves the problem of poor pickup effect near the boundary of the range, and improves the quality of the audio signal.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a method, device, terminal and storage medium for processing an audio signal. Background Art

[0002] At present, the terminal can pick up the audio signal in the target direction through the microphone. The implementation method of this process is: the audio signal of the surrounding environment picked up is processed by the beam forming method, and then the audio signal within the target pickup range of the target direction is obtained, so as to achieve the purpose of picking up the audio signal in the target direction.

[0003] In the related art, the minimum variance distortion-free response method is usually used to pick up the audio signal in the target direction. That is, under the constraint that the desired signal is distortion-free, a set of beam coefficients is designed to form a beam to pick up the audio signal in the target direction and suppress noise in other directions.

[0004] However, in the above method, the sound pickup effect is poor near the boundary of the target sound pickup range, there is a certain damage to the audio signal, and the noise suppression in other directions is not good, resulting in the audio obtained near the boundary of the target sound pickup range being unclear and of poor quality. Summary of the invention

[0005] The embodiments of the present application provide a method, device, terminal and storage medium for processing an audio signal, which processes the picked-up audio signal through a processing coefficient, and the processing coefficient can enhance the audio signal within the target sound pickup range and suppress the audio signal outside the target sound pickup range, so that the audio signal within the target sound pickup range can be enhanced and the audio signal outside the target sound pickup range can be suppressed by this method, which can improve the problem of poor sound pickup effect near the boundary of the target sound pickup range, thereby improving the quality of the audio signal. The technical solution is as follows:

[0006] According to one aspect of an embodiment of the present application, a method for processing an audio signal is provided, the method comprising:

[0007] Based on the beam coefficient, the first audio signal is processed to obtain a second audio signal, where the first audio signal is an audio signal obtained by sound pickup by the audio collection component, and the beam coefficient is used to indicate a target sound pickup range;

[0008] Determining a processing coefficient based on the first audio signal and the second audio signal, wherein the processing coefficient is used to enhance the audio signal within the sound pickup range and filter the audio signal outside the sound pickup range;

[0009] The second audio signal is processed based on the processing coefficient to obtain a third audio signal.

[0010] In a possible implementation manner, determining a processing coefficient based on the first audio signal and the second audio signal includes:

[0011] determining a first audio feature and a second audio feature, the first audio feature corresponding to the first audio signal and the second audio feature corresponding to the second audio signal;

[0012] The processing coefficient is determined based on the first audio feature and the second audio feature.

[0013] In another possible implementation manner, determining the processing coefficient based on the first audio feature and the second audio feature includes:

[0014] If the sound pickup direction of the second audio signal is within the target sound pickup range, determining a ratio of the first audio feature to the second audio feature to obtain the processing coefficient;

[0015] If the sound pickup direction of the second audio signal is not within the target sound pickup range, the processing coefficient is determined to be zero.

[0016] In another possible implementation manner, determining the first audio feature and the second audio feature includes:

[0017] determining a first spectral feature and a second spectral feature, the first spectral feature corresponding to the first audio signal, the second spectral feature corresponding to the second audio signal;

[0018] Determining the first audio feature based on the first frequency spectrum feature;

[0019] Based on the second frequency spectrum feature, the second audio feature is determined.

[0020] In another possible implementation manner, determining the first audio feature based on the first frequency spectrum feature includes:

[0021] Determine the first frequency spectrum feature as the first audio feature; or perform Mel filtering on the first frequency spectrum feature to obtain the first audio feature;

[0022] The determining the second audio feature based on the second frequency spectrum feature includes:

[0023] The second frequency spectrum feature is determined as the second audio feature; or, the second frequency spectrum feature is subjected to Mel filtering to obtain the second audio feature.

[0024] In another possible implementation manner, determining a processing coefficient based on the first audio signal and the second audio signal includes:

[0025] The first audio signal and the second audio signal are input into a coefficient determination model, and the processing coefficient is output. The coefficient determination model is used to determine the processing coefficient based on the two audio signals.

[0026] In another possible implementation manner, the processing the second audio signal based on the processing coefficient to obtain a third audio signal includes:

[0027] A product of the processing coefficient and the second audio signal is determined to obtain the third audio signal.

[0028] According to one aspect of an embodiment of the present application, there is provided a device for processing an audio signal, the device comprising:

[0029] A first processing module, configured to process a first audio signal based on a beam coefficient to obtain a second audio signal, wherein the first audio signal is an audio signal obtained by sound pickup by an audio collection component, and the beam coefficient is used to indicate a target sound pickup range;

[0030] a determination module, configured to determine a processing coefficient based on the first audio signal and the second audio signal, wherein the processing coefficient is used to enhance the audio signal within the sound pickup range and filter the audio signal outside the sound pickup range;

[0031] The second processing module is used to process the second audio signal based on the processing coefficient to obtain a third audio signal.

[0032] In a possible implementation, the determining module includes:

[0033] a first determining unit, configured to determine a first audio feature and a second audio feature, wherein the first audio feature corresponds to the first audio signal, and the second audio feature corresponds to the second audio signal;

[0034] The second determining unit is configured to determine the processing coefficient based on the first audio feature and the second audio feature.

[0035] In another possible implementation, the second determination unit is configured to determine a ratio of the first audio feature to the second audio feature to obtain the processing coefficient if the pickup direction of the second audio signal is within the target pickup range; and determine that the processing coefficient is zero if the pickup direction of the second audio signal is not within the target pickup range.

[0036] In another possible implementation, the first determination unit is configured to determine a first spectral feature and a second spectral feature, where the first spectral feature corresponds to the first audio signal and the second spectral feature corresponds to the second audio signal; determine the first audio feature based on the first spectral feature; and determine the second audio feature based on the second spectral feature.

[0037] In another possible implementation, the first determination unit is configured to determine the first spectral feature as the first audio feature; or, perform Mel filtering on the first spectral feature to obtain the first audio feature; determine the second spectral feature as the second audio feature; or, perform Mel filtering on the second spectral feature to obtain the second audio feature.

[0038] In another possible implementation, the determining module includes:

[0039] The third determination unit is configured to input the first audio signal and the second audio signal into a coefficient determination model and output the processing coefficient, wherein the coefficient determination model is used to determine the processing coefficient based on the two audio signals.

[0040] In another possible implementation manner, the second processing module is configured to determine a product of the processing coefficient and the second audio signal to obtain the third audio signal.

[0041] According to one aspect of an embodiment of the present application, a terminal is provided, which includes one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement a method for processing an audio signal in any possible implementation manner as described above.

[0042] According to one aspect of an embodiment of the present application, a storage medium is provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor to implement a method for processing an audio signal in any possible implementation manner as described above.

[0043] According to one aspect of an embodiment of the present application, a computer program product is provided, wherein the computer program product comprises at least one program code, wherein the at least one program code is loaded and executed by a processor, so that the computer implements a method for processing an audio signal in any possible implementation manner as described above.

[0044] In an embodiment of the present application, a method, device, terminal and storage medium for processing an audio signal are provided. The picked-up audio signal is processed by a processing coefficient, and the processing coefficient can enhance the audio signal within the target sound pickup range and suppress the audio signal outside the target sound pickup range. Thus, by this method, the audio signal within the target sound pickup range can be enhanced and the audio signal outside the target sound pickup range can be suppressed, which can improve the problem of poor sound pickup effect near the boundary of the target sound pickup range and improve the quality of the audio signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 It is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0047] Figure 2 is a flow chart of a method for processing an audio signal provided by an exemplary embodiment of the present application;

[0048] Figure 3 is a flow chart of a method for processing an audio signal provided by an exemplary embodiment of the present application;

[0049] Figure 4 is a spatial spectrum gain diagram of a theoretically picked-up audio signal provided by an exemplary embodiment of the present application;

[0050] Figure 5 is a spatial spectrum gain diagram of an audio signal picked up in practice provided by an exemplary embodiment of the present application;

[0051] Figure 6 is a schematic diagram of a method for processing an audio signal provided by an exemplary embodiment of the present application;

[0052] Figure 7 is a structural schematic diagram of an audio signal processing device provided by an exemplary embodiment of the present application;

[0053] Figure 8 It is a structural diagram of a terminal provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical scheme and advantages of the present application clearer, the implementation mode of the present application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, a first audio signal may be referred to as a second audio signal, and similarly, a second audio signal may be referred to as a first audio signal without departing from the scope of the application.

[0056] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application, such as Figure 1 As shown, the implementation environment includes a terminal 101 and a server 102. The terminal 101 and the server 102 are connected via a wireless or wired network.

[0057] Optionally, the terminal 101 is any type of terminal such as a smart phone, a tablet computer or a smart speaker, etc. The server 102 is a server, or a server cluster composed of several servers, or a cloud computing service center.

[0058] The terminal 101 is installed with an application provided by the server 102, and the terminal 101 can realize functions such as data transmission and message interaction through the application. Optionally, the application is an application in the operating system of the terminal 101, or an application provided by a third party. For example, the application is a microphone application; wherein the application at least has a collection function, and of course, the application also has other functions, such as a filtering function, an enhancement function, etc.

[0059] In some embodiments, the terminal 101 collects audio signals from all directions in the surrounding environment and sends the audio signals to the server 102. The server 102 processes the audio signals to obtain audio signals in the target pickup range. In other embodiments, the terminal 101 may also obtain audio signals in the target pickup range locally.

[0060] The audio signal processing method provided in the embodiment of the present application can be applied to any scenario of acquiring an audio signal in a target direction.

[0061] The first scenario is applied to telephone conference scenarios. For example, the two parties on a call conduct a telephone conference through a mobile phone. The mobile phone obtains audio signals from the surrounding environment. When someone in the meeting speaks, the mobile phone can accurately obtain the audio signal in the direction of this person and transmit the audio signal to the other party. The people on the other party can clearly hear what this person is saying.

[0062] The second scenario is applied to the audio signal control terminal to execute the target operation scenario. For example, the user wants to change the channel of the smart TV or adjust the volume of the smart TV. The smart TV can accurately obtain the audio signal in the direction of the user and then execute the corresponding operation in the audio signal.

[0063] It should be noted that the embodiments of the present application are only illustrated by taking the example of obtaining an audio signal in a target direction in a telephone conference, or obtaining an audio signal in a target direction when an audio signal control terminal performs a target operation, and do not limit the scenario of the audio signal processing method of the present application.

[0064] Figure 2 1 is a flow chart of a method for processing an audio signal provided by an embodiment of the present application. The embodiment of the present application is executed by a terminal, and the method comprises the following steps:

[0065] Step 201: The terminal processes a first audio signal based on a beam coefficient to obtain a second audio signal, where the first audio signal is an audio signal picked up by an audio collection component, and the beam coefficient is used to indicate a target sound pickup range.

[0066] Step 202: The terminal determines a processing coefficient based on the first audio signal and the second audio signal, where the processing coefficient is used to enhance the audio signal within the sound pickup range and filter the audio signal outside the sound pickup range.

[0067] Step 203: The terminal processes the second audio signal based on the processing coefficient to obtain a third audio signal.

[0068] In an embodiment of the present application, the terminal processes the picked up audio signal through a processing coefficient, and the processing coefficient can enhance the audio signal within the target pickup range and suppress the audio signal outside the target pickup range. Thus, through this method, the audio signal within the target pickup range can be enhanced and the audio signal outside the target pickup range can be suppressed. This can improve the problem of poor pickup effect near the boundary of the target pickup range, thereby improving the quality of the audio signal.

[0069] Figure 3 1 is a flow chart of a method for processing an audio signal provided by an embodiment of the present application. The embodiment of the present application is executed by a terminal, and the method comprises the following steps:

[0070] Step 301: The terminal processes a first audio signal based on a beam coefficient to obtain a second audio signal.

[0071] Among them, the beam coefficient is used to indicate the target pickup range, including multiple values, each of which is the weight of the audio signal in the corresponding direction. The terminal can obtain the audio signal within the target pickup range through the beam coefficient. First, the terminal needs to determine the beam coefficient, which is mainly determined by a beamforming algorithm. For example, the beamforming algorithm can be an MVDR (Minimum Variance Distortionless Response) algorithm. Under the constraint that the desired signal is distortion-free, the output variance is minimized, and the audio signal in the target direction is lossless. The beam coefficient is determined, and step 301 is executed through the beam coefficient. In an embodiment of the present application, the beamforming algorithm can also be an LCMV (Linearly Constrained Minimum Variance) algorithm, a DSB (Delay and Sum Beamforming) algorithm, etc., which is not specifically limited here.

[0072] The terminal includes an audio collection component that can collect audio signals from all directions in the surrounding environment. However, the user only needs the audio signal from one direction, and the audio signals from other directions interfere with the audio signal in this direction, making it impossible for the user to obtain the desired audio signal clearly and accurately. Therefore, the terminal needs to process the collected audio signal to obtain the audio signal required by the user. In the embodiment of the present application, the audio collection component can be any component for collecting audio signals, such as a microphone, a microphone, and a headset, included in the terminal, and is not specifically limited here.

[0073] Among them, the first audio signal is an audio signal picked up by the audio collection component, and the audio signal includes audio signals in various directions in the surrounding environment. The second audio signal is an audio signal in the target direction in the surrounding environment, and the target direction is located in the target pickup range. The terminal uses the beam coefficient to perform weighted processing on the audio signals in various directions in the first audio signal, and obtains the audio signal in the target pickup range where the target direction is located, that is, obtains the second audio signal.

[0074] For example, see Figure 4, the horizontal axis is the pickup direction, ranging from 0 to 180 degrees, and the vertical axis is the spatial gain value of the audio signal, ranging from 0 to 1. The user needs an audio signal in a 90-degree direction. The terminal uses the beam coefficient to pick up the audio signal with a pickup direction of 90 degrees and a beam width of 60 degrees to obtain a second audio signal. The theoretical pickup range (target pickup range) is 60 to 120 degrees, but in practice, for example, see Figure 5 , Figure 5 and Figure 4 The horizontal and vertical coordinates are the same and will not be repeated here. The ideal effect cannot be achieved at the boundary of the pickup range, that is, the actual pickup range is larger than the target pickup range, and the terminal picks up some noise outside the target pickup range, that is, there is noise in the second audio signal obtained by the terminal, and the quality is poor.

[0075] Due to the poor sound pickup effect, the actual sound pickup range is larger than the target sound pickup range, and the audio signal picked up near the boundary of the target sound pickup range has noise, resulting in that if the user speaks in the direction near the boundary of the target sound pickup range, the terminal cannot obtain a clear audio signal, thereby failing to achieve the user's purpose. Therefore, it is necessary to perform the following processing on the obtained audio signal to improve the quality of the audio signal.

[0076] Step 302: The terminal determines a first audio feature and a second audio feature.

[0077] The first audio feature corresponds to the first audio signal, and the second audio feature corresponds to the second audio signal. The terminal can obtain the spectral features of the first audio signal and the spectral features of the second audio signal. Accordingly, the steps for the terminal to determine the first audio feature and the second audio feature are: the terminal determines the first spectral feature and the second spectral feature, the first spectral feature corresponds to the first audio signal, and the second spectral feature corresponds to the second audio signal; based on the first spectral feature, the first audio feature is determined; based on the second spectral feature, the second audio feature is determined. For example, the terminal can obtain the spectral features of the first audio signal and the spectral features of the second audio signal through Fourier transform.

[0078] The terminal determines the first audio feature based on the first spectrum feature, and determines the second audio feature based on the second spectrum feature, which can be divided into the following multiple methods, including:

[0079] In the first manner, the terminal determines the first audio feature based on the first spectrum feature, and determines the second audio feature based on the second spectrum feature. The terminal determines the first spectrum feature as the first audio feature, and determines the second spectrum feature as the second audio feature.

[0080] In the embodiment of the present application, the terminal directly uses the frequency spectrum feature of the first audio signal as the first audio feature, and uses the frequency spectrum feature of the second audio signal as the second audio feature, which is simple to operate.

[0081] In the second method, after obtaining the spectral features of the first audio signal and the spectral features of the second audio signal, the terminal can further obtain the corresponding MFCC (Mel Frequency Cepstrum Coefficient) features as the corresponding audio features, that is, the MFCC features of the first audio signal are used as the first audio features, and the MFCC features of the second audio signal are used as the second audio features. Correspondingly, the steps for the terminal to determine the first audio features and the second audio features are: the terminal performs Mel filtering on the first spectral features to obtain the first audio features; and performs Mel filtering on the second spectral features to obtain the second audio features.

[0082] It should be noted that after the terminal performs Mel filtering on the first audio feature and the second audio feature, it also needs to perform discrete cosine transform on the obtained features, so as to obtain the MFCC features of the first audio signal and the MFCC features of the second audio signal.

[0083] In an embodiment of the present application, the terminal uses the MFCC feature of the first audio signal as the first audio feature and the MFCC feature of the second audio signal as the second audio feature, which is more in line with the human ear response characteristics, thereby improving the accuracy of the determined audio feature.

[0084] In the third method, after obtaining the spectral characteristics of the first audio signal and the spectral characteristics of the second audio signal, the terminal can further obtain the corresponding FBank (Filter Bank) characteristics as the corresponding audio characteristics, that is, the FBank characteristics of the first audio signal are used as the first audio characteristics, and the FBank characteristics of the second audio signal are used as the second audio characteristics. Correspondingly, the steps for the terminal to determine the first audio characteristics and the second audio characteristics are: the terminal performs Mel filtering on the first spectral characteristics to obtain the first audio characteristics; and performs Mel filtering on the second spectral characteristics to obtain the second audio characteristics. Different from the above-mentioned method of obtaining MFCC features, this method does not require discrete cosine transform in the process of obtaining FBank features.

[0085] In an embodiment of the present application, the terminal uses the FBank feature of the first audio signal as the first audio feature, and uses the FBank feature of the second audio signal as the second audio feature. Compared with the method of obtaining the MFCC feature, the amount of calculation is less and the running consumption is less.

[0086] In a fourth manner, after obtaining the frequency spectrum characteristics of the first audio signal and the frequency spectrum characteristics of the second audio signal, the terminal may further convert the frequency spectrum of the audio signal into an energy spectrum, and use the energy distribution in the energy spectrum as the corresponding audio characteristic, that is, use the energy distribution of the first audio signal as the first audio characteristic, and use the energy distribution of the second audio signal as the second audio characteristic. Accordingly, the steps for the terminal to determine the first audio characteristic and the second audio characteristic are: the terminal obtains the energy distribution of the first audio signal based on the first frequency spectrum characteristics to obtain the first audio characteristic; and obtains the energy distribution of the second audio signal based on the second frequency spectrum characteristics to obtain the second audio characteristic.

[0087] In an embodiment of the present application, the terminal uses the energy distribution in the energy spectrum of the first audio signal as the first audio feature, and uses the energy distribution in the energy spectrum of the second audio signal as the second audio feature. Compared with the method of obtaining MFCC features and the method of obtaining FBank features, the amount of calculation is less and the running consumption is less.

[0088] Step 303: The terminal determines a processing coefficient based on the first audio feature and the second audio feature.

[0089] The terminal has a poor sound pickup effect near the boundary of the target sound pickup range, resulting in the second audio signal obtained at this position being unclear and of poor quality. Further processing of the second audio signal is required. The terminal can first determine the processing coefficient to complete the subsequent processing process. The processing coefficient is used to enhance the audio signal within the sound pickup range and filter the audio signal outside the sound pickup range.

[0090] In a possible implementation, the terminal can determine the processing coefficient through the sound pickup direction and the target sound pickup range. When the terminal determines that the sound pickup direction is within the target sound pickup range, the terminal determines the processing coefficient based on the first audio feature and the second audio feature. When the terminal determines that the sound pickup direction is not within the target sound pickup range, the terminal directly determines the processing coefficient. Accordingly, the terminal determines the processing coefficient based on the first audio feature and the second audio feature in the following steps: if the sound pickup direction of the second audio signal is within the target sound pickup range, the terminal determines the ratio of the first audio feature to the second audio feature to obtain the processing coefficient; if the sound pickup direction of the second audio signal is not within the target sound pickup range, the terminal determines that the processing coefficient is zero. In which, since the audio signal is attenuated during the processing process, the ratio of the first audio feature to the second audio feature is a value greater than 1.

[0091] For example, see Figure 5The terminal determines that the target pickup range is 60 degrees to 120 degrees. When the pickup direction is within the target pickup range, the terminal uses the ratio of the first audio feature to the second audio feature as the processing coefficient. When the pickup direction is not within the target pickup range, that is, the pickup direction is within the range of 0 degrees to 60 degrees or within the range of 120 degrees to 180 degrees, the terminal uses zero as the processing coefficient.

[0092] In the embodiment of the present application, the terminal determines the processing coefficient through the sound pickup direction and the target sound pickup range, which is beneficial to the subsequent pickup of the audio signal near the boundary of the target sound pickup range.

[0093] In another possible implementation, the terminal can determine the processing coefficient through a coefficient determination model, and the process includes:

[0094] The terminal inputs the first audio signal and the second audio signal into a coefficient determination model, the coefficient determination model is used to determine the actual sound pickup range corresponding to the second audio signal, determines the processing coefficient based on the relationship between the target sound pickup range and the actual sound pickup range, and outputs the processing coefficient.

[0095] The coefficient determination model determines the processing coefficient based on the relationship between the target pickup range and the actual pickup range. The process includes: if the actual pickup range and the target pickup range are determined to be the same, the coefficient determination module determines the processing coefficient to be 1. If the actual pickup direction corresponding to the second audio signal is within the target pickup range, the coefficient determination model determines the ratio of the first audio feature to the second audio feature to obtain the processing coefficient; if the actual pickup direction corresponding to the second audio signal is not within the target pickup range, the coefficient determination model determines the processing coefficient to be 0. The actual pickup direction corresponding to the second audio signal is within the actual pickup range.

[0096] The coefficient determination model is used to determine the processing coefficient based on the two audio signals. The model can extract the first audio feature according to the input first audio signal and extract the second audio feature according to the second audio signal. This method is the same as step 302 and will not be repeated here. In the embodiment of the present application, the coefficient determination model can be a deep learning model, which is not specifically limited here.

[0097] In the embodiment of the present application, the terminal determines the processing coefficient through a coefficient determination model, and can quickly obtain the processing coefficient, thereby improving the processing efficiency of the audio signal.

[0098] One point that needs to be explained is that the coefficient determination model is obtained by training with a sample audio signal. Accordingly, the training process of the coefficient determination model is as follows: the terminal obtains a first sample audio signal and a second sample audio signal, the second sample audio signal is obtained by processing the first sample audio signal based on a beam coefficient, and the second sample audio signal is annotated with a first relationship, the first relationship is used to represent the relationship between an actual sample sound pickup range and a target sample sound pickup range corresponding to the second sample audio signal, the first sample audio signal and the second sample audio signal are input into the initial coefficient determination model, the initial coefficient determination model determines a processing coefficient based on the first sample audio signal and the second sample audio signal, the processing coefficient corresponds to a second relationship, the second relationship is the relationship between the sound pickup range corresponding to the second sample audio signal and the target sample sound pickup range, and the sound pickup range corresponding to the second sample audio signal is obtained by the initial coefficient determination model based on the second sample audio signal, and the initial coefficient determination model is updated based on the first relationship and the second relationship to obtain the coefficient determination model.

[0099] Another point to be explained is that after the terminal determines the processing coefficient, it can directly execute step 304; it can also first determine whether the processing coefficient is 1; if the processing coefficient is 1, the second audio signal is not processed, thereby reducing operating consumption; if the processing coefficient is not 1, step 304 is executed.

[0100] Step 304: The terminal processes the second audio signal based on the processing coefficient to obtain a third audio signal.

[0101] Among them, the third audio signal is an audio signal within the target pickup range. The terminal determines the product of the processing coefficient and the second audio signal to obtain the third audio signal. When the pickup direction of the second audio signal is within the target pickup range, the processing coefficient obtained by the terminal is the ratio of the first audio feature to the second audio feature, which is a value greater than 1. At this time, multiplying the processing coefficient by the second audio signal can enhance the audio signal within the target pickup range; when the pickup direction of the second audio signal is not within the target pickup range, the processing coefficient obtained by the terminal is zero. At this time, multiplying the processing coefficient by the second audio signal can suppress the audio signal outside the target pickup range. The third audio signal thus obtained is the audio signal within the target pickup range.

[0102] Near the boundary of the target sound pickup range, the terminal also enhances the audio signal within the target sound pickup range and suppresses the audio signal outside the target sound pickup range. Thus, the terminal can improve the sound pickup effect at the boundary of the target sound pickup range, that is, reduce the damage of the audio signal within the boundary and the residual noise outside the boundary. Figure 5 , the terminal can improve the sound pickup effect of the spatial spectrum gain value near the boundary of the target sound pickup range within a certain range (for example, 0.2-0.8).

[0103] In an embodiment of the present application, the terminal processes the picked up audio signal through a processing coefficient, and the processing coefficient can enhance the audio signal within the target pickup range and suppress the audio signal outside the target pickup range. Thus, through this method, the audio signal within the target pickup range can be enhanced and the audio signal outside the target pickup range can be suppressed. This can improve the problem of poor pickup effect near the boundary of the target pickup range and improve the quality of the audio signal.

[0104] Figure 6 1 is a flow chart of a method for processing an audio signal provided in an embodiment of the present application. The embodiment of the present application is executed by a server, and the method comprises the following steps:

[0105] Step 601: The terminal obtains a first audio signal.

[0106] Step 602: The terminal sends a first audio signal to the server.

[0107] Step 603: The server processes the first audio signal based on the beam coefficient to obtain a second audio signal.

[0108] Step 604: The server determines a first audio feature and a second audio feature.

[0109] Step 605: The server determines a processing coefficient based on the first audio feature and the second audio feature.

[0110] Step 606: The server processes the second audio signal based on the processing coefficient to obtain a third audio signal.

[0111] Steps 603-606 are the same as steps 301-304 and are not described in detail here.

[0112] Step 607: The server sends a third audio signal to the terminal.

[0113] In an embodiment of the present application, the server processes the picked up audio signal through a processing coefficient, and the processing coefficient can enhance the audio signal within the target pickup range and suppress the audio signal outside the target pickup range. Thus, through this method, the audio signal within the target pickup range can be enhanced and the audio signal outside the target pickup range can be suppressed. This can improve the problem of poor sound pickup effect near the boundary of the target pickup range and improve the quality of the audio signal.

[0114] Figure 7 is a schematic diagram of the structure of an audio signal processing device provided in an embodiment of the present application, see Figure 7 , the device comprises:

[0115] A first processing module 701 is used to process the first audio signal based on the beam coefficient to obtain a second audio signal, where the first audio signal is an audio signal obtained by sound collection by the audio collection component, and the beam coefficient is used to indicate a target sound collection range;

[0116] A determination module 702, configured to determine a processing coefficient based on the first audio signal and the second audio signal, wherein the processing coefficient is used to enhance the audio signal within the sound pickup range and filter the audio signal outside the sound pickup range;

[0117] The second processing module 703 is used to process the second audio signal based on the processing coefficient to obtain a third audio signal.

[0118] In a possible implementation, the determining module 701 includes:

[0119] A first determining unit, configured to determine a first audio feature and a second audio feature, wherein the first audio feature corresponds to the first audio signal and the second audio feature corresponds to the second audio signal;

[0120] The second determining unit is used to determine the processing coefficient based on the first audio feature and the second audio feature.

[0121] In another possible implementation, the second determination unit is configured to determine a ratio of the first audio feature to the second audio feature to obtain a processing coefficient if the pickup direction of the second audio signal is within a target pickup range; and to determine that the processing coefficient is zero if the pickup direction of the second audio signal is not within the target pickup range.

[0122] In another possible implementation, the first determination unit is used to determine a first spectral feature and a second spectral feature, the first spectral feature corresponds to the first audio signal, and the second spectral feature corresponds to the second audio signal; determine the first audio feature based on the first spectral feature; and determine the second audio feature based on the second spectral feature.

[0123] In another possible implementation, the first determination unit is used to determine the first spectral feature as the first audio feature; or, perform Mel filtering on the first spectral feature to obtain the first audio feature; determine the second spectral feature as the second audio feature; or, perform Mel filtering on the second spectral feature to obtain the second audio feature.

[0124] In another possible implementation, the determining module 702 includes:

[0125] The third determination unit is used to input the first audio signal and the second audio signal into a coefficient determination model and output a processing coefficient. The coefficient determination model is used to determine the processing coefficient based on the two audio signals.

[0126] In another possible implementation, the second processing module 703 is configured to determine a product of the processing coefficient and the second audio signal to obtain a third audio signal.

[0127] In an embodiment of the present application, in an embodiment of the present application, the picked-up audio signal is processed by a processing coefficient, and the processing coefficient can enhance the audio signal within the target pickup range and suppress the audio signal outside the target pickup range. Thus, by this method, the audio signal within the target pickup range can be enhanced, and the audio signal outside the target pickup range can be suppressed, which can improve the problem of poor sound pickup effect near the boundary of the target pickup range, thereby improving the quality of the audio signal.

[0128] Figure 8 The structure block diagram of a terminal 800 provided by an exemplary embodiment of the present application is shown. The terminal 800 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Group Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4), a laptop computer or a desktop computer. The terminal 800 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0129] Typically, the terminal 800 includes a processor 801 and a memory 802 .

[0130] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0131] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one instruction, which is used to be executed by the processor 801 to implement the audio signal processing method provided in the method embodiment of the present application.

[0132] In some embodiments, the terminal 800 may also optionally include: a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802 and the peripheral device interface 803 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 803 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 804, a touch display screen 805, a camera 806 camera assembly 806, an audio circuit 807, a positioning assembly 808 and a power supply 809.

[0133] The peripheral device interface 803 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 may be implemented on a separate chip or circuit board, which is not limited in the present application.

[0134] The radio frequency circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 804 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 804 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 804 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0135] The display screen 805 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen 805 is a touch display screen, the display screen 805 also has the ability to collect touch signals on the surface or above the surface of the display screen 805. The touch signal can be input to the processor 801 as a control signal for processing. At this time, the display screen 805 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 805 can be one, and the front panel of the terminal 800 is set; in other embodiments, the display screen 805 can be at least two, which are respectively set on different surfaces of the terminal 800 or are folded; in some embodiments, the display screen 805 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 800. Even, the display screen 805 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 805 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0136] The camera assembly 806 is used to capture images or videos. Optionally, the camera assembly 806 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 806 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0137] The audio circuit 807 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 801 for processing, or input them into the radio frequency circuit 804 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 800. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 807 may also include a headphone jack.

[0138] The positioning component 808 is used to locate the current geographical location of the terminal 800 to implement navigation or LBS (Location Based Service). The positioning component 808 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system, or Russia's European Union's Galileo system.

[0139] The power supply 809 is used to power various components in the terminal 800. The power supply 809 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 809 includes a rechargeable battery, the rechargeable battery can be a battery that supports wired charging or a wireless charging battery. A wired charging battery is a battery that is charged through a wired line, and a wireless charging battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0140] In some embodiments, the terminal 800 further includes one or more sensors 810 , including but not limited to: an acceleration sensor 811 , a gyroscope sensor 812 , a pressure sensor 813 , a fingerprint sensor 814 , an optical sensor 815 , and a proximity sensor 816 .

[0141] The acceleration sensor 811 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 800. For example, the acceleration sensor 811 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 801 can control the touch display screen 805 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 811. The acceleration sensor 811 can also be used for collecting game or user motion data.

[0142] The gyroscope sensor 812 can detect the body direction and rotation angle of the terminal 800, and the gyroscope sensor 812 can cooperate with the acceleration sensor 811 to collect the user's 3D actions on the terminal 800. The processor 801 can implement the following functions based on the data collected by the gyroscope sensor 812: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0143] The pressure sensor 813 can be set on the side frame of the terminal 800 and / or the lower layer of the touch display screen 805. When the pressure sensor 813 is set on the side frame of the terminal 800, it can detect the user's holding signal of the terminal 800, and the processor 801 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 813. When the pressure sensor 813 is set on the lower layer of the touch display screen 805, the processor 801 controls the operability controls on the UI interface according to the user's pressure operation on the touch display screen 805. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0144] The fingerprint sensor 814 is used to collect the user's fingerprint, and the processor 801 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 814, or the fingerprint sensor 814 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 801 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 814 can be set on the front, back, or side of the terminal 800. When a physical button or a manufacturer logo is set on the terminal 800, the fingerprint sensor 814 can be integrated with the physical button or the manufacturer logo.

[0145] The optical sensor 815 is used to collect the ambient light intensity. In one embodiment, the processor 801 can control the display brightness of the touch display screen 805 according to the ambient light intensity collected by the optical sensor 815. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 805 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 805 is reduced. In another embodiment, the processor 801 can also dynamically adjust the shooting parameters of the camera assembly 806 according to the ambient light intensity collected by the optical sensor 815.

[0146] The proximity sensor 816, also called a distance sensor, is usually arranged on the front panel of the terminal 800. The proximity sensor 816 is used to collect the distance between the user and the front of the terminal 800. In one embodiment, when the proximity sensor 816 detects that the distance between the user and the front of the terminal 800 is gradually decreasing, the processor 801 controls the touch display screen 805 to switch from the screen-on state to the screen-off state; when the proximity sensor 816 detects that the distance between the user and the front of the terminal 800 is gradually increasing, the processor 801 controls the touch display screen 805 to switch from the screen-off state to the screen-on state.

[0147] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the terminal 800, and the terminal 800 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0148] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a terminal to implement the audio signal processing method in the above-mentioned embodiment. The computer-readable storage medium can be a memory. For example, the computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0149] In an exemplary embodiment, a computer program product is further provided. The computer program product includes at least one program code. The at least one program code is loaded and executed by a processor, so that the computer implements the audio signal processing method in the above embodiment.

[0150] In an exemplary embodiment, the computer program involved in the embodiments of the present application may be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected by a communication network. Multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain system.

[0151] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0152] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing an audio signal, characterized in that: The method comprises: Based on the beam coefficient, the first audio signal is processed to obtain a second audio signal, where the first audio signal is an audio signal obtained by sound pickup by the audio collection component, and the beam coefficient is used to indicate a target sound pickup range; determining a first audio feature and a second audio feature, the first audio feature corresponding to the first audio signal and the second audio feature corresponding to the second audio signal; If the sound pickup direction of the second audio signal is within the target sound pickup range, determining a ratio of the first audio feature to the second audio feature to obtain a processing coefficient; If the pickup direction of the second audio signal is not within the target pickup range, the processing coefficient is determined to be zero; the processing coefficient is used to enhance the audio signal within the pickup range and filter the audio signal outside the pickup range; The second audio signal is processed based on the processing coefficient to obtain a third audio signal.

2. The method according to claim 1, characterized in that: The determining of the first audio feature and the second audio feature comprises: determining a first spectral feature and a second spectral feature, the first spectral feature corresponding to the first audio signal, the second spectral feature corresponding to the second audio signal; Determining the first audio feature based on the first frequency spectrum feature; Based on the second frequency spectrum feature, the second audio feature is determined.

3. The method according to claim 2, characterized in that: The determining the first audio feature based on the first frequency spectrum feature includes: Determine the first frequency spectrum feature as the first audio feature; or perform Mel filtering on the first frequency spectrum feature to obtain the first audio feature; The determining the second audio feature based on the second frequency spectrum feature includes: The second frequency spectrum feature is determined as the second audio feature; or, the second frequency spectrum feature is subjected to Mel filtering to obtain the second audio feature.

4. The method according to any one of claims 1 to 3, characterized in that: The step of processing the second audio signal based on the processing coefficient to obtain a third audio signal includes: A product of the processing coefficient and the second audio signal is determined to obtain the third audio signal.

5. An audio signal processing device, characterized in that: The device comprises: A first processing module, configured to process a first audio signal based on a beam coefficient to obtain a second audio signal, wherein the first audio signal is an audio signal obtained by sound pickup by an audio collection component, and the beam coefficient is used to indicate a target sound pickup range; a determination module, configured to determine a first audio feature and a second audio feature, wherein the first audio feature corresponds to the first audio signal, and the second audio feature corresponds to the second audio signal; if the sound pickup direction of the second audio signal is within the target sound pickup range, then determining a ratio of the first audio feature to the second audio feature to obtain a processing coefficient; if the sound pickup direction of the second audio signal is not within the target sound pickup range, then determining that the processing coefficient is zero; the processing coefficient is used to enhance the audio signal within the sound pickup range and filter the audio signal outside the sound pickup range; The second processing module is used to process the second audio signal based on the processing coefficient to obtain a third audio signal.

6. A terminal, characterized in that: The terminal includes one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the audio signal processing method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: At least one program code is stored in the storage medium, and the at least one program code is loaded and executed by the processor to implement the audio signal processing method according to any one of claims 1 to 4.

8. A computer program product, characterized in that The computer program product comprises at least one program code, and the at least one program code is loaded and executed by a processor to implement the audio signal processing method according to any one of claims 1 to 4.

Images