Audio signal processing method and device, terminal equipment and storage medium

Abstract

The invention relates to an audio signal processing method and device, terminal equipment and a storage medium. The method comprises the steps that a first audio signal is acquired through a first acquisition unit of the terminal equipment, and a second audio signal is acquired through a second acquisition unit of the terminal equipment; the distance difference between the first acquisition unit and the first sound source and the distance difference between the second acquisition unit and the first sound source are within a first range; the distance difference between the first acquisition unit and the second acquisition unit and the second sound source is greater than the maximum value of the first range, and the first sound source is a target acquisition object; superposing the first audio signal and the second audio signal in a time domain to obtain a superposed audio signal; and filtering the noise audio signal with the amplitude smaller than a preset value in the superposed audio signal to obtain a target audio signal of the target collection object. The interference of the sound emitted by the sound source on the required sound is reduced, the required audio signal is obtained, the noise is reduced, and the quality of the audio signal is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of audio signal processing, and in particular to an audio signal processing method, apparatus, terminal equipment and storage medium. Background Art

[0002] With the development of audio technology, more and more terminal devices have an audio signal processing function. The terminal devices with the audio signal processing function usually have a microphone, through which the audio signal of the environment where the terminal device is located can be collected.

[0003] In some specific application scenarios, when there are multiple sound sources in the environment where the terminal device is located, the microphone will collect various sounds in the current environment. For example, if multiple people are talking in the current scene, the microphone can collect everyone's voice. Some sounds may be required according to business needs, while some sounds may not be required by the business scenario. Summary of the invention

[0004] The present disclosure provides an audio signal processing method, an apparatus, a terminal device and a storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an audio signal processing method, which is applied to a terminal device, and the method comprises: obtaining a first audio signal by collecting a first collection unit of the terminal device and obtaining a second audio signal by collecting a second collection unit of the terminal device; wherein the distance difference between the first collection unit and the second collection unit and a first sound source is within a first range; the distance difference between the first collection unit and the second collection unit and the second sound source is greater than a maximum value of the first range, and the first sound source is a target collection object; superimposing the first audio signal and the second audio signal in a time domain to obtain a superimposed audio signal; and filtering a noise audio signal whose amplitude is less than a preset value in the superimposed audio signal to obtain a target audio signal of the target collection object.

[0006] In one embodiment, the terminal device includes a rectangular frame and a back shell; the rectangular frame includes: a group of long sides parallel to each other and a group of short sides parallel to each other; the long sides and the short sides are connected end to end; when the terminal device is in a horizontal screen posture, the collection surface of the first collection unit and the collection surface of the second collection unit are located at different positions of the same long side; when the terminal device is in a vertical screen posture, the collection surface of the first collection unit and the collection surface of the second collection unit are located on different sides of the rectangular frame; when the terminal device is in a tilted state or a vertical posture, the collection surface of the first collection unit is located on the rectangular frame, and the collection surface of the second collection unit is located on the back shell.

[0007] In one embodiment, the method further includes: determining a collection scenario; setting audio processing parameters of the terminal device according to the current collection scenario; wherein the audio processing parameters are at least used to determine a collection unit and / or collection parameters for audio collection in the collection scenario;

[0008] Among them, the collection scene includes a first scene; wherein, the first scene is a scene for audio collection of a single sound source; in the first scene, the audio processing parameters are at least used to determine that the first collection unit and the second collection unit collect audio and filter it.

[0009] In one embodiment, the audio processing parameters corresponding to the first scene include: the preset value; first direction information, used to indicate the sound collection direction of the first collection unit and the second collection unit; and a first noise reduction parameter, used to reduce ambient sound noise in the first scene.

[0010] In one embodiment, the collection scenario further includes: a second scenario, wherein in the second scenario, audio signals of multiple sound sources are collected by at least one audio collection unit in the terminal device.

[0011] In one embodiment, the audio processing parameters corresponding to the second scene include at least one of the following: collection unit information, used to determine the collection unit for collecting multiple sound sources; second direction information, used to collect the sound direction of the multiple sound sources; distance information, used to determine the collection distance of the sound source; second noise reduction parameters, used for ambient sound noise reduction of the second scene.

[0012] In one embodiment, the acquisition scenario further includes: a third scenario, wherein the third scenario is used to acquire audio based on an audio acquisition peripheral device independent of the terminal device.

[0013] In one embodiment, the audio processing parameters corresponding to the second scene include at least one of the following: first indication information, used to indicate the use of an audio acquisition peripheral connected to the terminal device to collect audio signals; a third noise reduction parameter, used to indicate turning off the environmental denoising function when using the acquisition unit on the terminal device for audio acquisition; second indication information, used to instruct the acquisition unit of the terminal device to collect ambient sound.

[0014] In one embodiment, the determining of the acquisition scenario includes: determining the acquisition scenario according to an input operation acting on the terminal device; or determining the acquisition scenario according to whether there is an audio acquisition peripheral device connected to the terminal device.

[0015] According to a third aspect of the embodiments of the present disclosure, there is provided an audio signal processing device, including:

[0016] A collection module, used to collect a first audio signal through a first collection unit and to collect a second audio signal through a second collection unit; wherein the distance difference between the first collection unit and the second collection unit and a first sound source is within a first range; the distance difference between the first collection unit and the second collection unit and a second sound source is greater than a maximum value of the first range, and the first sound source is a target collection object; a superposition module, used to superimpose the first audio signal and the second audio signal in a time domain to obtain a superimposed audio signal; a filtering module, used to filter a noise audio signal whose amplitude is less than a preset value in the superimposed audio signal to obtain a target audio signal of the target collection object.

[0017] According to a third aspect of the embodiments of the present disclosure, a terminal device is provided, including:

[0018] A processor and a memory for storing executable instructions that can be run on the processor, wherein: when the processor is used to run the executable instructions, the executable instructions execute the method described in any one of the above embodiments.

[0019] According to a fourth aspect of the embodiments of the present disclosure, a non-temporary computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0020] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0021] The audio signal processing method in the embodiment of the present disclosure collects different audio signals through different acquisition units, for example, the first acquisition unit collects the first audio signal, and the second acquisition unit collects the second audio signal. After the two sound sources make sounds, the two acquisition units can respectively collect the audio signals emitted by the two sound sources. Since the distance difference between the two acquisition units and the first sound source is within the first range, the difference in the time domain between the audio signals of the first sound source collected by the two acquisition units is small. After superimposing the audio signals of the first sound source collected by the two acquisition units, the audio signal can be enhanced and the amplitude can be increased. Since the distance difference between the two acquisition units and the second sound source is greater than the maximum value of the first range, the audio signals of the second sound source collected by the two acquisition units are greatly different in the time domain, and the amplitude after superposition will not increase significantly. According to the preset value, the noise signal with an amplitude less than the preset value in the superimposed audio signal can be filtered out, and the noise signal includes the audio signal emitted by the second sound source, so that the audio signal of the second sound is filtered out in the audio signal received by the two acquisition units, and the audio signal emitted by the first sound source is obtained. The interference of the sound emitted by the second sound source on the audio signal emitted by the required first sound source is reduced, and the target audio signal of the target acquisition object is obtained, the noise in the target audio signal is reduced, the quality of the target audio signal is improved, and thus the user experience is improved.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0024] Figure 1 is a schematic diagram of an audio signal processing method according to an exemplary embodiment;

[0025] Figure 2 is a schematic diagram showing the distribution of an audio signal acquisition unit according to an exemplary embodiment;

[0026] Figure 3 is a schematic diagram showing the distribution of another audio signal acquisition unit according to an exemplary embodiment;

[0027] Figure 4 is a schematic diagram showing a method of determining an application scenario according to an exemplary embodiment;

[0028] Figure 5 is a schematic diagram showing a method of determining a collection scene according to an exemplary embodiment;

[0029] Figure 6According to an exemplary embodiment, a method based on Figure 5 Determine the interface diagram of the collection scene;

[0030] Figure 7 is a schematic diagram of an audio signal processing device according to an exemplary embodiment;

[0031] Figure 8 is a schematic diagram of an audio signal processing method according to an exemplary embodiment;

[0032] Fig. 9 is a schematic diagram of an audio signal processing method according to an exemplary embodiment;

[0033] Fig.10 The present invention is a block diagram of a terminal device according to an exemplary embodiment. DETAILED DESCRIPTION

[0034] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0035] Voice over Internet Protocol (VoIP) calls based on IP have gradually replaced SIM card-based calls. VoIP has become the main way of information transmission in daily life and work, such as online meetings and chats on instant messaging tools. The quality of VoIP calls is an important evaluation of the call effect of terminal devices such as mobile phones or tablets. In real-life scenarios, call quality is often affected by the noise of the surrounding environment. For example, when a user participates in an online meeting, the user's voice will be transmitted to the receiving end together with the surrounding environment and the voices of others. The user at the receiving end will hear that the sound is very noisy, which reduces the call experience.

[0036] Normally, the audio collection units (such as microphones) on terminal devices such as tablets and mobile phones can perform 360-degree omnidirectional sound pickup. However, when used in a noisy environment, other sounds in the environment and the desired sound will be collected by the microphone of the terminal device. Even if the ambient sound is suppressed in the noise reduction algorithm of the terminal device, other sounds in the environment will still be retained, causing the desired sound to be interfered with or even covered by other sounds in the environment.

[0037] For example, when a user participates in a video conference at home, when the user speaks, after the microphone performs 360-degree omnidirectional sound pickup, the voices of other family members speaking nearby will also be recorded by the microphone and transmitted to the receiving end, causing the sound received by the receiving end to be noisy, affecting the user at the receiving end from obtaining the required sound, reducing the call quality, and also reducing the user's privacy.

[0038] refer to Figure 1 , is a schematic diagram of an audio signal processing method, which can be executed at least in a terminal device having an audio signal acquisition unit, and the terminal device can include a mobile terminal device and a fixed terminal device, that is, the execution subject of the method can at least include a mobile terminal device and a fixed terminal device. The mobile terminal device can include a mobile phone, a tablet computer, a wearable device, etc.

[0039] The audio signal processing method comprises:

[0040] S100: A first audio signal is obtained by collecting through a first collection unit of a terminal device, and a second audio signal is obtained by collecting through a second collection unit of the terminal device; wherein a distance difference between the first collection unit and the second collection unit and the first sound source is within a first range; and a distance difference between the first collection unit and the second collection unit and the second sound source is greater than a maximum value of the first range, and the first sound source is a target collection object.

[0041] S200: Superimpose the first audio signal and the second audio signal in the time domain to obtain a superimposed audio signal.

[0042] S300: Filter the noise audio signal whose amplitude is smaller than a preset value in the superimposed audio signal to obtain the target audio signal of the target acquisition object.

[0043] In the terminal device of this embodiment, the terminal device has multiple audio signal acquisition units, referred to as acquisition units, and the positions of the acquisition units are different. For example, 3, 4 or 5 acquisition units may be included, and the acquisition units may be located on the frame of the terminal device or on the back of the terminal device, that is, on the back shell. The terminal device may be a terminal device with a display screen, and the back here may be the side of the terminal device opposite to the display screen.

[0044] For S100, the audio signal acquisition unit may include a microphone, and the audio signal acquisition unit may also be called a pickup.

[0045] Exemplarily, collection unit 1 and collection unit 2 are located in the first frame of the terminal device, collection unit 3 is located in the second frame of the terminal device, and collection unit 4 is located on the back shell of the terminal device. The first frame can be perpendicular to the second frame, and the first frame can be parallel to the second frame.

[0046] refer to Figure 2 , is a schematic diagram of the distribution of a collection unit. The collection unit is Figure 2 In the example, an omnidirectional microphone is used. Figure 2 The front view of the terminal device is shown in FIG. 1 , where the front is the side where the display screen is located. The omnidirectional microphone mic1 and the omnidirectional microphone mic2 are located at different positions on the same frame of the terminal device. The terminal device has a rectangular frame, and the omnidirectional microphone mic4 is located on another frame of the terminal device. The two frames are perpendicular and are two sides of the rectangular frame of the terminal device.

[0047] Exemplarily, the collection surface of the omnidirectional microphone mic1 and the collection surface of the omnidirectional microphone mic2 are located on the same frame.

[0048] refer to Figure 3 , is a schematic diagram of the distribution of another type of collection unit. Figure 3 In the example, an omnidirectional microphone is used. Figure 3 The figure is a back view of the terminal device, that is, a schematic diagram of the back shell. The omnidirectional microphone mic3 is located on the back shell.

[0049] Exemplarily, the collecting surface of the omnidirectional microphone mic3 is located on the back shell.

[0050] Figure 2 and Figure 3 Also shown are a speaker, a camera (such as a front camera) and a USB interface.

[0051] In one embodiment, Figure 2 and Figure 3 is a schematic diagram of a terminal device in a horizontal screen posture. Figure 2 Rotate 90 degrees clockwise to get the vertical screen posture of the terminal device. Figure 3 Rotate 90 degrees counterclockwise to get the vertical screen posture of the terminal device.

[0052] In one embodiment, the terminal device is a tablet computer and a mobile phone. Figure 2 and Figure 3 A schematic diagram of a tablet computer.

[0053] There is a preset distance between two different collection units, and the preset distance can be determined according to the design of the terminal device. Figure 2 and Figure 3 There is a preset distance between the omnidirectional microphone mic1 and the omnidirectional microphone mic2, and the omnidirectional microphone mic1 and the omnidirectional microphone mic2 can divide the first frame into three equal parts. The omnidirectional microphone mic4 can be located in the middle of the second frame. The first frame is perpendicular to the second frame.

[0054] Exemplarily, the first acquisition unit may be Figure 2 and Figure 3 The omnidirectional microphone mic1 in the second acquisition unit can be Figure 2 and Figure 3 The omnidirectional microphone mic2 in the.

[0055] The sounds emitted by each sound source are collected respectively by two of the multiple audio signal collection units in the terminal device. The two collection units may not be fixed and may be determined according to the actual usage scenario.

[0056] For example, the terminal device Figure 2 In the state shown, the audio signal emitted by the first sound source can be collected by the omnidirectional microphone mic1 and the omnidirectional microphone mic2, and the audio signal emitted by the second sound source can be collected by the omnidirectional microphone mic1 and the omnidirectional microphone mic2. The omnidirectional microphone mic1 can be used as the first collection unit, and the omnidirectional microphone mic2 can be used as the second collection unit.

[0057] For another example, the omnidirectional microphone mic1 and the omnidirectional microphone mic4 collect audio signals from various sound sources respectively. The omnidirectional microphone mic1 can be used as a first collection unit, and the omnidirectional microphone mic4 can be used as a second collection unit.

[0058] For example, the terminal device Figure 3 In the state shown, any one of the omnidirectional microphone mic1, the omnidirectional microphone mic2 and the omnidirectional microphone mic4 serves as the first acquisition unit, and the omnidirectional microphone mic3 serves as the second acquisition unit.

[0059] The audio signal collected by the first collection unit is recorded as a first audio signal, and the audio signal collected by the second collection unit is recorded as a second audio signal.

[0060] For example, the second sound source is located at Figure 2 If the second sound source emits sound 1 on the left or right side of the terminal device, both the first acquisition unit (such as omnidirectional microphone mic1) and the second acquisition unit (such as omnidirectional microphone mic2) can collect the sound 1. The sound 1 collected by the first audio signal is recorded as the first audio signal, and the sound 1 collected by the second audio signal is recorded as the second audio signal.

[0061] The distances between the first acquisition unit and the second acquisition unit and the second sound source are different, the time when the sound 1 is transmitted to the first acquisition unit and the second acquisition unit is different, and the time when the first acquisition unit and the second acquisition unit receive the sound 1 is also different.

[0062] For another example, the first sound source is directed toward Figure 2The front side of the terminal device shown in the figure is at an equal distance from the omnidirectional microphone mic1 and the omnidirectional microphone mic2. Then the time for the sound 2 emitted by the first sound source to be transmitted to the omnidirectional microphone mic1 and the omnidirectional microphone mic2 is the same, and the time for the omnidirectional microphone mic1 and the omnidirectional microphone mic2 to collect the sound 2 is also the same.

[0063] Exemplarily, the sound source 2 may be a user of the terminal device.

[0064] After a sound source emits a sound, each collection unit on the terminal device can collect the sound. After different sound sources emit sounds, each collection unit can also collect the sounds emitted by each sound source. The sounds emitted by each sound source are within the collection range of the collection unit.

[0065] In one embodiment, there are two sound sources, a first sound source and a second sound source, and audio signals emitted by the two sound sources are collected by a first acquisition unit and a second acquisition unit. The first sound source is at a distance from the first acquisition unit and the second acquisition unit, respectively, and the distance difference between the two distances is within a first range. The first sound source is the target acquisition object, and the audio signal finally obtained is the audio signal emitted by the first sound source. The second sound source is at a distance from the first acquisition unit and the second acquisition unit, respectively, and the distance difference between the two distances is greater than the maximum value of the first range.

[0066] The first range can be determined according to actual use requirements, and can be a distance range in centimeters or millimeters, but the specific value is not limited.

[0067] The first audio signal can be collected by the first collection unit, and the second audio signal can be collected by the second collection unit. The first audio signal and the second audio signal can be audio signals from the same sound source. Exemplarily, it can be determined based on the spectrum analysis of the audio signal whether the first audio signal and the second audio signal are audio signals from the same sound source.

[0068] Exemplarily, the first audio signal and the second audio signal may be audio signals emitted by a first sound source, and the first audio signal and the second audio signal may also be audio signals emitted by a second sound source.

[0069] After the first audio signal and the second audio signal are collected, frequency spectrum information of the first audio signal in the time domain and frequency spectrum information of the second audio signal in the time domain may be determined.

[0070] The greater the distance difference between the sound source and the two acquisition units, the greater the difference in the time domain of the audio signals emitted by the sound source collected by the two acquisition units, and the farther the peaks of the amplitudes are. The smaller the distance difference between the sound source and the two acquisition units, the smaller the difference in the time domain of the audio signals emitted by the sound source collected by the two acquisition units, and the closer the peaks of the amplitudes are.

[0071] For S200, after obtaining the first audio signal and the second audio signal, the first audio signal and the second audio signal are superimposed in the time domain to obtain a superimposed audio signal. When the frequency spectrum information difference between the first audio signal and the second audio signal in the time domain is large, the amplitude overlap degree of the first audio signal and the second audio signal is low, the amplitude of the superimposed audio signal may be small, and the enhancement effect is small.

[0072] When the frequency spectrum information difference between the first audio signal and the second audio signal in the time domain is small, the amplitude overlap degree of the first audio signal and the second audio signal is high, the amplitude of the superimposed audio signal may be large, and the enhancement effect is large. The amplitude of the superimposed audio signal includes the maximum amplitude.

[0073] For S300, the superimposed audio signal is filtered by a preset value, and the audio signal whose amplitude of the superimposed audio signal is less than the preset value is filtered as a noise signal to obtain the target audio signal. In this way, the sound of the sound source due to the large distance difference between the sound source and the two collection units, such as the sound emitted by the sound source on the left or right side of the terminal device or the direction in which the rear shell faces, is filtered out, and the audio signal emitted by the target collection object is obtained, thereby reducing the noise in the obtained audio signal and improving the quality of the target audio signal.

[0074] Since the distance difference between the two acquisition units and the first sound source is within the first range, the audio signals of the first sound source collected by the two acquisition units have a small difference in the time domain. After superimposing the audio signals of the first sound source collected by the two acquisition units, the audio signal can be enhanced and the amplitude can be increased. Since the distance difference between the two acquisition units and the second sound source is greater than the maximum value of the first range, the audio signals of the second sound source collected by the two acquisition units have a large difference in the time domain, and the amplitude after superposition will not increase significantly. According to the preset value, the noise signal with an amplitude less than the preset value in the superimposed audio signal can be filtered out, and the noise signal includes the audio signal emitted by the second sound source. In this way, the audio signal of the second sound is filtered out in the audio signal received by the two acquisition units, and the audio signal emitted by the first sound source is obtained. The interference of the sound emitted by the second sound source on the audio signal emitted by the required first sound source is reduced, and the target audio signal of the target acquisition object is obtained, the noise in the target audio signal is reduced, and the quality of the target audio signal is improved, thereby improving the user experience.

[0075] In one embodiment, the terminal device includes a rectangular frame and a rear shell, the rectangular frame includes: a group of long sides parallel to each other and a group of short sides parallel to each other; the long sides and the short sides are connected end to end.

[0076] refer to Figure 2 and Figure 3 The largest rectangle shown is the frame, which includes two long sides parallel to each other and two short sides parallel to each other. The long sides and the short sides are connected to form a rectangular frame. Exemplarily, two can be a group.

[0077] refer to Figure 2 and Figure 3 , the terminal device is in a horizontal screen posture, when the terminal device is in a horizontal screen posture, the collection surface of the first collection unit and the collection surface of the second collection unit are located at different positions of the same long side. For example, Figure 2 The topmost frame shown is a long side frame, and the collection surface of the first collection unit and the second collection unit are located at different positions of the top frame, that is, the omnidirectional microphone mic1 and the omnidirectional microphone mic2 are located on the same long side frame.

[0078] Exemplarily, the omnidirectional microphone mic1 and the omnidirectional microphone mic2 may also be located on the bottom long side frame.

[0079] Exemplarily, the omnidirectional microphone mic1 and the omnidirectional microphone mic2 may be located on the frame of the long side and the short side, respectively.

[0080] For example, in actual application scenarios, such as when using instant messaging software for voice communication or video communication, the terminal device can be in a horizontal screen posture.

[0081] In one embodiment, when the terminal device is in a vertical screen posture, the collection surface of the first collection unit and the collection surface of the second collection unit are located on different sides of the rectangular frame.

[0082] Exemplarily, the collection surface of the first collection unit is located on the long side frame, and the collection surface of the second collection unit is located on the short side frame. Figure 2 The omnidirectional microphone mic1 is located on the long side frame, and the omnidirectional microphone mic4 is on the short side frame.

[0083] Exemplarily, the collection surface of the first collection unit is located on a first long side frame, and the collection surface of the second collection unit is located on another long side frame.

[0084] In one embodiment, when the terminal device is in a tilted state or a vertical posture, the collection surface of the first collection unit is located on the rectangular frame, and the collection surface of the second collection unit is located on the rear shell.

[0085] Exemplarily, the collection surface of the first collection unit is located on the frame of any long side, and the collection surface of the second collection unit is located on the back shell. Figure 3 The omnidirectional microphone mic1 is located on the long side frame, and the omnidirectional microphone mic3 is located on the back shell.

[0086] In one embodiment, reference Figure 4 , is a schematic diagram of determining an application scenario, the method further comprising:

[0087] S10, determining the collection scene;

[0088] S20, setting audio processing parameters of the terminal device according to the current acquisition scenario; wherein the audio processing parameters are at least used to determine an acquisition unit and / or acquisition parameters for audio acquisition in the acquisition scenario.

[0089] The collection scene includes a first scene; wherein the first scene is a scene for collecting audio from a single sound source;

[0090] In the first scenario, the audio processing parameter is at least used to determine whether the first acquisition unit and the second acquisition unit collect audio and perform filtering.

[0091] In S10, the collection scenario is determined, including:

[0092] The acquisition scenario is determined according to the input operation acting on the terminal device; or, the acquisition scenario is determined according to whether there is an audio acquisition peripheral connected to the terminal device.

[0093] Different input operations determine different collection scenarios, and different collection scenarios correspond to different audio processing parameters. When the audio processing parameters are different, the collection methods of the audio signals collected by the collection unit are different, the processing methods of the collected audio signals are also different, and the collection units that collect the audio signals are also different. The collection method can be determined according to the collection parameters, and when the collection parameters are different, the collection methods can be different.

[0094] The collection scenario can also be determined based on whether the terminal device is connected to an audio collection peripheral. If an audio collection peripheral is connected, a collection scenario related to the audio signal collected by the audio collection peripheral can be determined, such as a scenario of collecting audio signals through the audio collection peripheral.

[0095] refer to Figure 5 , which is a schematic diagram of an interface for determining a collection scene, refer to Figure 6 Based on Figure 5 Determine the interface diagram of the collection scene.

[0096] Figure 5The figure shows an interface diagram of a VoIP-based application in a terminal device, such as an application with a conference function, including a conference toolbox interface, which includes a call noise reduction control. After the application is started, the interface can be displayed. For example, the conference toolbox can also be an application, and the interface after opening is as follows Figure 5 As shown. According to the operation on the call noise reduction control, you can get Figure 6 The interface shown.

[0097] Figure 6 for Figure 5 A detailed diagram of the call noise reduction interface is shown. Figure 6 The interface for determining the acquisition scene is shown, including single-person scene, multi-person scene, microphone noise reduction scene, and speaker / headphone noise reduction scene. Figure 6 As shown, the call noise reduction interface includes a control for whether the microphone noise reduction is turned on and a scene control, and the collection scene can be determined according to the input operation. The first scene can include a single-person scene, and the first scene is a scene for collecting audio from a single sound source. In the single-person scene, the microphone picks up the sound directly in front of the screen.

[0098] After selecting a single-person acquisition scenario, in the first scenario, the audio processing parameters are at least used to determine that the first acquisition unit and the second acquisition unit collect audio and perform filtering, and the target audio signal can be determined according to S100 to S300.

[0099] In one embodiment, the first scene corresponding audio processing parameter includes at least one of the following:

[0100] Preset value for filtering the superimposed audio signal;

[0101] The first direction information is used to indicate the sound collection direction of the first collection unit and the second collection unit;

[0102] A first noise reduction parameter, used for ambient sound noise reduction in a first scenario;

[0103] The acquisition unit information is used to determine whether to acquire the first acquisition unit or the second acquisition unit.

[0104] The preset value can be determined according to the actual scenario and can be an integer or a decimal in decibels.

[0105] The first direction information may include the directions of 30 degrees to the left and right of the front of the terminal device, indicating that the sound collection direction is a total range of 60 degrees formed by the first direction information indication. Of course, it can also be other directions, indicating other angles deviating from the front.

[0106] The first noise reduction parameters may include parameters related to noise reduction, such as maximum noise reduction depth, noise reduction coefficient, noise reduction intensity and noise reduction radius.

[0107] The collection unit information may be identification information of the first collection unit and the second collection unit.

[0108] In one embodiment, the acquisition scenario further includes:

[0109] A second scenario, wherein in the second scenario, audio signals of multiple sound sources are collected by at least one audio collection unit in a terminal device.

[0110] The second scene is different from the first scene. The second scene collects audio signals emitted by multiple sound sources through one or more collection units.

[0111] For example, according to the effect on Figure 6 The operation of the multi-person scene on the interface shown can determine that the collection scene is the second scene. The second scene includes Figure 6 The multiplayer scene shown.

[0112] Exemplarily, in the second scenario, it may not be necessary to execute S100 to S300.

[0113] In one embodiment, the audio processing parameters corresponding to the second scene include at least one of the following:

[0114] Collection unit information, used to determine the collection unit for collecting multiple sound sources;

[0115] Second direction information, used for collecting directions of sound from the multiple sound sources;

[0116] Distance information, used to determine the collection distance of the sound source;

[0117] The second noise reduction parameter is used for reducing the ambient sound noise of the second scene. The content included in the first noise reduction parameter may be referred to.

[0118] The collection unit information may be identification information of a collection unit for collecting multiple sound sources.

[0119] The second direction information may be similar to the first direction information, and the specific direction indicated may be different. The second direction information may indicate the collection of audio signals emitted by multiple sound sources within 360 degrees around the terminal device, and the collection direction is 360 degrees.

[0120] The distance information can be determined based on the performance parameters of the collection unit and / or the call requirements, such as an area with a radius of 5 meters with the terminal device as the center. The maximum collection distance is 5 meters.

[0121] The audio signal may be collected according to the audio processing parameters corresponding to the second scene.

[0122] In one embodiment, the acquisition scenario further includes:

[0123] The third scenario, wherein the third scenario is used to collect audio based on an audio collection peripheral device independent of the terminal device.

[0124] refer to Figure 6 In the headphone noise reduction collection mode shown, the collection scenario after the speaker / headphone noise reduction switch is turned on is the third scenario. The third scenario is different from the first and second scenarios. The audio signal is not collected by the collection unit on the terminal device, but by an audio collection peripheral independent of the terminal device.

[0125] For example, a wireless headset independent of a terminal device collects audio signals around the headset, and then the headset sends the collected audio signals to the terminal device.

[0126] In one embodiment, the audio processing parameters corresponding to the second scene include at least one of the following:

[0127] The first indication information is used to instruct to use an audio acquisition peripheral connected to the terminal device to collect audio signals;

[0128] The third noise reduction parameter is used to indicate turning off the environment noise reduction function when using the acquisition unit on the terminal device to collect audio;

[0129] The second indication information is used to instruct the collection unit of the terminal device to collect ambient sound.

[0130] When the audio processing parameters corresponding to the second scenario include the first indication information, it can be determined whether to use an audio acquisition peripheral connected to the terminal device to collect the audio signal according to the first indication information.

[0131] The third noise reduction parameter may refer to the content included in the first noise reduction parameter.

[0132] When the audio processing parameters corresponding to the second scene include the second indication information, the ambient sound can be collected by a collection unit of the terminal device according to the second indication information.

[0133] The first indication information and the second indication information may exist at the same time, and the audio signal is collected by the audio collection peripheral device and the collection unit at the same time.

[0134] In one embodiment, reference Figure 7 , is a schematic diagram of an audio signal processing device, the device comprising:

[0135] A collection module 1 is used to collect a first audio signal through a first collection unit and to collect a second audio signal through a second collection unit; wherein the distance difference between the first collection unit and the second collection unit and a first sound source is within a first range; the distance difference between the first collection unit and the second collection unit and the second sound source is greater than a maximum value of the first range, and the first sound source is a target collection object;

[0136] A superposition module 2, configured to superimpose the first audio signal and the second audio signal in a time domain to obtain a superimposed audio signal;

[0137] The filtering module 3 is used to filter the noise audio signal whose amplitude is less than a preset value in the superimposed audio signal to obtain the target audio signal of the target acquisition object.

[0138] In one embodiment, the terminal device comprises a rectangular frame and a rear shell; the rectangular frame comprises: a group of long sides parallel to each other and a group of short sides parallel to each other; the long sides and the short sides are connected end to end;

[0139] When the terminal device is in a horizontal screen posture, the collection surface of the first collection unit and the collection surface of the second collection unit are located at different positions of the same long side;

[0140] When the terminal device is in a vertical screen posture, the collection surface of the first collection unit and the collection surface of the second collection unit are located on different sides of the rectangular frame;

[0141] When the terminal device is in a tilted state or a vertical posture, the collection surface of the first collection unit is located on the rectangular frame, and the collection surface of the second collection unit is located on the rear shell.

[0142] In one embodiment, the apparatus further comprises:

[0143] A scene determination module, used to determine the acquisition scene;

[0144] A parameter determination module, used to set the audio processing parameters of the terminal device according to the current acquisition scenario; wherein the audio processing parameters are at least used to determine the acquisition unit and / or acquisition parameters for audio acquisition in the acquisition scenario;

[0145] The acquisition scene includes a first scene; wherein the first scene is a scene for collecting audio from a single sound source;

[0146] In the first scenario, the audio processing parameters are at least used to determine that the first acquisition unit and the second acquisition unit collect audio and perform filtering.

[0147] In one embodiment, the first scene corresponding audio processing parameter includes at least one of the following:

[0148] The preset value;

[0149] First direction information, used to indicate the sound collection directions of the first collection unit and the second collection unit;

[0150] The first noise reduction parameter is used for ambient sound noise reduction in the first scenario.

[0151] In one embodiment, the acquisition scenario further includes:

[0152] A second scenario, wherein, in the second scenario, audio signals of multiple sound sources are collected by at least one audio collection unit in the terminal device.

[0153] In one embodiment, the audio processing parameters corresponding to the second scene include at least one of the following:

[0154] Collection unit information, used to determine the collection unit for collecting multiple sound sources;

[0155] Second direction information, used for collecting directions of sound from the multiple sound sources;

[0156] Distance information, used to determine the collection distance of the sound source;

[0157] The second noise reduction parameter is used for ambient sound noise reduction in the second scene.

[0158] In one embodiment, the acquisition scenario further includes:

[0159] A third scenario, wherein the third scenario is used to collect audio based on an audio collection peripheral device independent of the terminal device.

[0160] In one embodiment, the audio processing parameters corresponding to the second scene include at least one of the following:

[0161] The first indication information is used to instruct to use an audio acquisition peripheral connected to the terminal device to collect an audio signal;

[0162] The third noise reduction parameter is used to indicate turning off the environment noise reduction function when the collection unit on the terminal device is used for audio collection;

[0163] The second indication information is used to instruct the collection unit of the terminal device to collect ambient sound.

[0164] In one embodiment, the scene determination module is further configured to:

[0165] determining the acquisition scenario according to an input operation applied to the terminal device; or

[0166] The acquisition scenario is determined according to whether there is an audio acquisition peripheral device connected to the terminal device.

[0167] It should be noted that the “first” and “second” in the embodiments of the present disclosure are only for the convenience of description and distinction and have no other specific meanings.

[0168] In one embodiment, Figure 8 and Fig. 9 The figure is a schematic diagram of two different audio signal processing methods. The call toolbox is a VoIP-based application and ADSP is an advanced digital signal processor. When the VoIP app starts a call, the pickup mode defaults to a multi-person scenario and sends audio signal processing parameters, such as the remote_record_mode parameter, to the network audio hardware abstraction layer (AudioHAL). After receiving the audio signal processing parameters, AudioHAL Figure 2 and Figure 3 The microphone combination shown is used to configure the VoIP uplink audio path and the corresponding audio signal processing parameters ( Figure 8 and Fig. 9 in the algorithm).

[0169] For example, during a meeting, if the user needs to suppress the surrounding sound, they can click Figure 7 In the single-person scenario shown, the processing parameters of the audio signal in the single-person scenario, such as the remote_record_mode parameter, will be sent. After receiving the audio signal processing parameters, AudioHAL will adjust the microphone combination, turn on the microphone on the back of the tablet, and modify the processing parameters of the audio signal so that the tablet only picks up the sound within a certain angle range from the front and suppresses the sound from the back and surroundings.

[0170] The terminal's gyroscope senses whether the tablet is in landscape or portrait mode. In a single-person scenario, a total of four microphones collect sound data and send it to the voice algorithm for processing. The principle of the algorithm here can refer to S100 to S300, and algorithms with the same principle can be included in the algorithm. If the current mode is landscape, the sound on both sides and the back of the tablet is suppressed according to the data of the omnidirectional microphone mic1, omnidirectional microphone mic2, and omnidirectional microphone mic3. If the current mode is portrait, only the sound on the long side where the omnidirectional microphone mic1 and omnidirectional microphone mic2 are located can be suppressed. At this time, the audio data of the omnidirectional microphone mic4 on the short side can also be read to suppress the sound on the other side, optimizing the sound pickup effect of the portrait mode in a single-person scenario.

[0171] Fig.10 1 is a block diagram of a terminal device according to an exemplary embodiment. For example, the terminal device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0172] Reference Fig.10The terminal device may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0173] The processing component 802 generally controls the overall operation of the terminal device, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0174] The memory 804 is configured to store various types of data to support operations on the terminal device. Examples of such data include instructions for any application or method operating on the terminal device, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0175] The power component 806 provides power to various components of the terminal device. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device.

[0176] The multimedia component 808 includes a screen that provides an output interface between the terminal device and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the terminal device is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0177] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the terminal device is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0178] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0179] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the terminal device. For example, the sensor assembly 814 can detect the open / closed state of the terminal device, the relative positioning of components, such as the display and keypad of the terminal device, and the sensor assembly 814 can also detect the position change of the terminal device or a component of the terminal device, the presence or absence of user contact with the terminal device, the orientation or acceleration / deceleration of the terminal device, and the temperature change of the terminal device. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0180] The communication component 816 is configured to facilitate wired or wireless communication between the terminal device and other devices. The terminal device can access a wireless network based on a communication standard, such as Wi-Fi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0181] In an exemplary embodiment, the terminal device may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0182] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0183] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An audio signal processing method, characterized in that: The method comprises: A first audio signal is acquired by a first acquisition unit of a terminal device, and a second audio signal is acquired by a second acquisition unit of the terminal device; wherein the distance difference between the first acquisition unit and the second acquisition unit and the first sound source is within a first range; the distance difference between the first acquisition unit and the second acquisition unit and the second sound source is greater than a maximum value of the first range, and the first sound source is a target acquisition object; Superimposing the first audio signal and the second audio signal in the time domain to obtain a superimposed audio signal; The noise audio signal whose amplitude is less than a preset value in the superimposed audio signal is filtered to obtain the target audio signal of the target acquisition object.

2. The method according to claim 1, characterized in that: The terminal device comprises a rectangular frame and a rear shell; the rectangular frame comprises: a group of long sides parallel to each other and a group of short sides parallel to each other; the long sides and the short sides are connected end to end; When the terminal device is in a horizontal screen posture, the collection surface of the first collection unit and the collection surface of the second collection unit are located at different positions of the same long side; When the terminal device is in a vertical screen posture, the collection surface of the first collection unit and the collection surface of the second collection unit are located on different sides of the rectangular frame; When the terminal device is in a tilted state or a vertical posture, the collection surface of the first collection unit is located on the rectangular frame, and the collection surface of the second collection unit is located on the rear shell.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Determine the collection scenario; According to the current acquisition scenario, the audio processing parameters of the terminal device are set; wherein the audio processing parameters are at least used to determine the acquisition unit and / or acquisition parameters for audio acquisition in the acquisition scenario; The acquisition scene includes a first scene; wherein the first scene is a scene for collecting audio from a single sound source; In the first scenario, the audio processing parameters are at least used to determine that the first acquisition unit and the second acquisition unit collect audio and perform filtering.

4. The method according to claim 3, characterized in that The audio processing parameter corresponding to the first scene includes at least one of the following: The preset value; First direction information, used to indicate the sound collection directions of the first collection unit and the second collection unit; The first noise reduction parameter is used for ambient sound noise reduction in the first scenario.

5. The method according to claim 3, characterized in that: The collection scenario also includes: A second scenario, wherein, in the second scenario, audio signals of multiple sound sources are collected by at least one audio collection unit in the terminal device.

6. The method according to claim 5, characterized in that The audio processing parameters corresponding to the second scene include at least one of the following: Collection unit information, used to determine the collection unit for collecting multiple sound sources; Second direction information, used for collecting directions of sound from the multiple sound sources; Distance information, used to determine the collection distance of the sound source; The second noise reduction parameter is used for reducing ambient sound noise in the second scene.

7. The method according to claim 3, characterized in that The collection scenario also includes: A third scenario, wherein the third scenario is used to collect audio based on an audio collection peripheral device independent of the terminal device.

8. The method according to claim 7, characterized in that The audio processing parameters corresponding to the second scene include at least one of the following: The first indication information is used to instruct to use an audio acquisition peripheral connected to the terminal device to collect an audio signal; The third noise reduction parameter is used to indicate turning off the environment noise reduction function when the collection unit on the terminal device is used for audio collection; The second indication information is used to instruct the collection unit of the terminal device to collect ambient sound.

9. The method according to claim 3, characterized in that: The determining of the acquisition scenario includes: determining the acquisition scenario according to an input operation applied to the terminal device; or The acquisition scenario is determined according to whether there is an audio acquisition peripheral device connected to the terminal device.

10. An audio signal processing device, characterized in that: include: A collection module, configured to obtain a first audio signal by collecting through a first collection unit and obtain a second audio signal by collecting through a second collection unit; wherein the distance difference between the first collection unit and the second collection unit and the first sound source is within a first range; the distance difference between the first collection unit and the second collection unit and the second sound source is greater than a maximum value of the first range, and the first sound source is a target collection object; a superposition module, configured to superpose the first audio signal and the second audio signal in a time domain to obtain a superimposed audio signal; The filtering module is used to filter the noise audio signal whose amplitude is less than a preset value in the superimposed audio signal to obtain the target audio signal of the target acquisition object.

11. A terminal device, characterized in that: include: A processor and a memory for storing executable instructions capable of running on the processor, wherein: When the processor is used to run the executable instructions, the executable instructions execute the method described in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method described in any one of claims 1 to 9.

Images