Audio processing system, method and electronic device

Abstract

The present application provides an audio processing system, method and electronic device. The electronic device side can provide the user with an ambient sound control function; if the current ambient sound control mode is a noise reduction mode, the audio signal sent by the electronic device to the earphone includes a reverse wave audio signal of the ambient audio, and the sound wave signal corresponding to the reverse wave audio signal is opposite in phase to the sound wave signal of the ambient audio that directly enters the user's ear, and the ambient sound can be offset. If the current ambient sound control mode is a transparent mode, the audio signal sent by the electronic device to the earphone includes a forward wave audio signal of the ambient audio, and the sound wave signal corresponding to the forward wave audio signal is the same in phase as the sound wave signal of the ambient audio that directly enters the user's ear, and the ambient sound can be enhanced. In this way, even if the earphones worn by the user do not have an ambient sound control function, the user can still enjoy a quiet audio listening experience and can clearly hear the ambient sound without taking off the earphones.

Description

Technical Field

[0001] The present application relates to the field of intelligent terminal technology, and in particular to an audio processing system, method and electronic device. Background Art

[0002] After a wired or wireless connection is established between an electronic device (such as a mobile phone, tablet computer, etc.) and a headset, the user can make calls through the headset or listen to audio played by the electronic device through the headset; in specific implementation, for application scenarios where the user listens to audio, if the user is in a noisy live environment, the user will not only hear the media audio (such as song audio, book audio, etc.) played by the headphone speaker, but also the external environmental noise will enter the user's ears, and the environmental noise will inevitably affect the user's media audio listening effect. Summary of the invention

[0003] In order to solve the above technical problems, the embodiments of the present application provide an audio processing system, method and electronic device. In this system, the electronic device side can provide the user with an ambient sound control function; if the current ambient sound control mode is a noise reduction mode, the audio signal sent by the electronic device to the earphone includes a reverse wave audio signal of the ambient audio, and the sound wave signal corresponding to the reverse wave audio signal is opposite in phase to the sound wave signal of the ambient audio directly entering the user's ear, and the ambient sound can be offset at this time. If the current ambient sound control mode is a transparent mode, the audio signal sent by the electronic device to the earphone includes a forward wave audio signal of the ambient audio, and the sound wave signal corresponding to the forward wave audio signal is the same in phase as the sound wave signal of the ambient audio directly entering the user's ear, and the ambient sound can be enhanced at this time. In this way, even if the earphones worn by the user do not have an ambient sound control function, the user can still enjoy a quiet audio listening experience and can clearly hear the ambient sound without taking off the earphones.

[0004] In the first aspect, an embodiment of the present application provides an audio processing system. The system includes an electronic device and a headset, the electronic device includes a microphone, the control mode of the electronic device for ambient sound includes a noise reduction mode and a transparent mode, and the headset does not have an ambient sound control function; the electronic device and the headset are connected by wire or wirelessly; the electronic device is used to obtain a media audio signal of a preset application; and the ambient audio signal is collected in real time through the microphone of the electronic device; the electronic device is also used to generate a first audio signal according to the current control mode and the ambient audio signal; the first audio signal and the media audio signal are processed by signal superposition to obtain a second audio signal, and the second audio signal is sent to the headset; wherein, when the current control mode is the noise reduction mode, the first audio signal includes a reverse wave audio signal corresponding to the ambient audio signal, and when the current control mode is the transparent mode, the first audio signal includes a forward wave audio signal corresponding to the ambient audio signal; the headset is used to receive and play the received second audio signal.

[0005] Exemplarily, the electronic device may be a mobile phone, a tablet computer, etc.; the earphone may be a wireless earphone or a wired earphone.

[0006] Exemplarily, the preset application may be an audio playback application, and the audio playback application may include any one or more of the following: a music application, a video application, and an audiobook application. Correspondingly, the media audio signal may include any one or more of the following: a song audio stream, a movie audio stream, and a book audio stream.

[0007] Exemplarily, the control mode of the electronic device for ambient sound can be called the ambient sound control mode. The current control mode refers to the ambient sound control mode that has been turned on during the generation process of the first audio signal. The first audio signal can be the target ambient audio signal mentioned in the following embodiments. The second audio signal can be the target superimposed audio signal mentioned in the following embodiments.

[0008] In this way, even if the headphones worn by the user do not have an ambient sound control function, the user can still enjoy a quiet audio listening experience and can clearly hear the ambient sound without taking off the headphones. That is to say, when the current control mode of the electronic device is set to the noise reduction mode, the reverse wave audio signal of the ambient audio is generated to offset the ambient sound that directly enters the user's ears, thereby providing the user with a quiet audio listening environment; when the current control mode of the electronic device is set to the transparent mode, the forward wave audio signal of the ambient audio is generated to enhance the ambient sound that directly enters the user's ears, and the user can clearly hear the surrounding ambient sound without taking off the headphones.

[0009] According to the first aspect, in the system, the electronic device is specifically used to generate a third audio signal according to the current control mode and the ambient audio signal; wherein, when the current control mode is the noise reduction mode, the third audio signal includes an initial reverse wave audio signal corresponding to the ambient audio signal, and when the current control mode is the transparency mode, the third audio signal includes an initial forward wave audio signal corresponding to the ambient audio signal; the electronic device is specifically used to increase or decrease the amplitude of the initial reverse wave audio signal according to the noise reduction amplitude of the noise reduction mode to obtain a reverse wave audio signal; or, increase or decrease the amplitude of the initial forward wave audio signal according to the transparency amplitude of the transparency mode to obtain a forward wave audio signal.

[0010] Exemplarily, the noise reduction amplitude refers to the amplitude of the noise reduction effect determined by the electronic device in response to the user's first adjustment trigger operation. The transparency amplitude refers to the amplitude of the transparency effect determined by the electronic device in response to the user's second adjustment trigger operation.

[0011] Exemplarily, the third audio signal may be the initial ambient audio signal mentioned in the following embodiments. The initial ambient audio signal may be understood as the reverse wave audio signal before the noise reduction strength is adjusted, or the forward wave audio signal before the transparency strength is adjusted. Correspondingly, the target ambient audio signal may be understood as the reverse wave audio signal after the noise reduction strength is adjusted, or the forward wave audio signal after the transparency strength is adjusted.

[0012] In this way, by providing users with a strength adjustment control for the ambient sound control effect under the current control mode, users can use the adjustment control to personalize the strength of the noise reduction or transparency effect based on the actual audio listening effect and the current environment. In this way, users can not only choose to turn on the noise reduction mode or transparency mode, but also adjust the strength of the noise reduction or transparency effect at any time.

[0013] According to the first aspect, or any implementation of the first aspect above, in the system, the electronic device is also specifically used to analyze the ambient sound type according to the ambient audio signal to determine the ambient sound type if the current control mode is the noise reduction mode; and determine the adjustable parameter information of the preset reverse wave generation algorithm according to the ambient sound type; and use the preset reverse wave generation algorithm to generate an initial reverse wave audio signal according to the adjustable parameter information and the ambient audio signal.

[0014] Exemplarily, the ambient sound type may be divided according to the frequency band type of the ambient audio, and the ambient sound type may include any one of a high frequency band type, a medium frequency band type, and a low frequency band type. The adjustable parameter information may include a weight coefficient of the frequency band to which the ambient audio signal belongs.

[0015] In this way, for the case where the current control mode is set to the noise reduction mode, the ambient sound type of the ambient audio signal is identified, and the adjustable parameters of the preset reverse wave generation algorithm are adaptively adjusted in combination with the identified ambient sound type, and then based on the preset reverse wave generation algorithm after the parameter adjustment, a reverse wave audio signal of the ambient audio is generated, thereby improving the generation accuracy of the reverse wave audio signal of the ambient audio, and then ensuring the offsetting effect of the reverse sound wave signal of the ambient audio contained in the sound wave signal played by the speaker of the earphone and the sound wave signal of the ambient audio directly entering the user's ear.

[0016] According to the first aspect, or any implementation of the first aspect above, in the system, the electronic device is also used to identify the scene type of the user's environment; and obtain target association information corresponding to the scene type; if it is determined according to the target association information that the first mode switching condition is currently met, the current control mode is switched from the noise reduction mode to the transparency mode; if it is determined according to the target association information that the second mode switching condition is currently met, the current control mode is switched from the transparency mode to the noise reduction mode.

[0017] Exemplarily, the scene types of the user's environment may include travel scenes by public transportation, office scenes, etc.; the target association information corresponding to different scene types may be different. The target association information may include at least one of the recognition result of the associated application corresponding to the current scene type (referred to as the first recognition result for short), and the recognition result obtained by performing signal recognition on the ambient audio signal (referred to as the second recognition result for short). Among them, different scene types correspond to different associated applications, and therefore, different scene types correspond to different first recognition results. Different scene types correspond to different ambient audio recognition dimensions, and therefore, different scene types correspond to different second recognition results.

[0018] Exemplarily, whether mode switching is required is determined based on the first recognition result and the second recognition result; for the first type of scenario, if both the first recognition result and the second recognition result satisfy the ambient sound enhancement trigger condition, it is determined that the first mode switching condition is currently satisfied; if both the first recognition result and the second recognition result satisfy the ambient sound shielding trigger condition, it is determined that the second mode switching condition is currently satisfied. For the second type of scenario, if either the first recognition result or the second recognition result satisfies the ambient sound enhancement trigger condition, it is determined that the first mode switching condition is currently satisfied; if either the first recognition result or the second recognition result satisfies the ambient sound shielding trigger condition, it is determined that the second mode switching condition is currently satisfied.

[0019] In this way, by first identifying the scene type of the user's current environment, and then identifying whether the current control mode needs to be switched based on the target-related information corresponding to the scene type, automatic switching of the current control mode can be achieved for the user, eliminating the need for the user to perform manual mode operations multiple times, and improving the intelligence of switching the current control mode.

[0020] According to the first aspect, or any implementation of the first aspect above, in the system, the electronic device is also used to determine the valid audio signal and the invalid audio signal in the ambient audio signal according to the scene type if the current control mode is the transparent mode; wherein the first audio signal includes a forward wave audio signal corresponding to the valid audio signal and a reverse wave audio signal corresponding to the invalid audio signal.

[0021] For example, different scene types may correspond to different effective audio signals, that is, the effective ambient audio that the user really wants to hear clearly in the transparent mode is different. For example, the effective ambient audio corresponding to the scene of traveling by public transportation may include the station announcement voice; for another example, the effective ambient audio corresponding to the scene of working in the office may include the voice of colleagues communicating with the user.

[0022] In this way, for the case where the current control mode is set to the transparency mode, in the process of generating the first audio signal, the forward wave audio signal of all ambient audio is not directly generated, but the scene type of the user's current environment is combined to identify which of the ambient audio signals are valid audio signals and which are invalid audio signals; then, the forward wave audio signal corresponding to the valid audio signal is generated, so that the user can clearly hear the valid ambient audio; and the reverse wave audio signal corresponding to the invalid audio signal is generated, so that the user can also block the invalid ambient audio in the transparency mode.

[0023] According to the first aspect, or any implementation of the first aspect above, in the system, the electronic device also includes an application processor and a digital signal processor; the application processor is used to obtain a media audio signal of a preset application, and to determine a mode identifier of a current control mode; the application processor is also used to drive the microphone to collect ambient audio signals in real time; and to send first indication information and a mode identifier to the digital signal processor; the digital signal processor is used to create an audio playback link and a first feedback link according to the first indication information; and to obtain the media audio signal from the application processor through the audio playback link; and to obtain the ambient audio signal from the microphone through the first feedback link; the digital signal processor is also used to generate a first audio signal according to the mode identifier and the ambient audio signal.

[0024] Exemplarily, the first feedback link may be an ambient sound feedback link, the data acquisition port of the ambient sound feedback link is the data transmission port of a microphone of the electronic device, and the data sending port of the ambient sound feedback link is a signal processing module in a digital signal processor.

[0025] In this way, when it is detected that the headset does not have an ambient sound control function, not only the application processor drives the microphone to collect ambient audio signals, but also the application processor controls the digital signal processor to simultaneously create an audio playback link and an ambient sound feedback link; the digital signal processor then obtains the media audio signal through the audio playback link, and directly obtains the ambient audio signal from the microphone through the ambient sound feedback link, and then the digital signal processor generates a first audio signal based on the media audio signal and the ambient audio signal. Since the generation process of the second audio signal is performed by the digital signal processor, and the acquisition process of the ambient audio signal is directly obtained by the digital signal processor from the microphone. Therefore, the data flow of the ambient audio signal only needs to be from the microphone to the digital signal processor, without passing through the hardware abstraction layer and the application layer program framework layer in the software architecture involved in the application processor, which can greatly shorten the acquisition time of the ambient audio signal, thereby improving the correspondence between the sound wave signal corresponding to the first audio signal played by the headset and the sound wave signal of the ambient audio directly entering the user's ear, and further improving the user's audio listening experience.

[0026] According to the first aspect, or any implementation of the first aspect above, in the system, the application processor is further used to send a microphone turn-on instruction to the headset when it is detected that the electronic device enters pocket mode; and send a second indication information to the digital signal processor; the headset is further used to turn on the microphone of the headset in response to the microphone turn-on instruction to collect the ambient audio signal in real time through the microphone; and transmit the ambient audio signal to the audio transmission interface of the electronic device; the digital signal processor is further used to destroy the first feedback link according to the second indication information, and create a second feedback link to obtain the ambient audio signal from the audio transmission interface through the second feedback link.

[0027] Exemplarily, the second feedback link may be an ambient sound feedback link newly created by the digital signal processor when the pocket mode is detected, the data acquisition port of the ambient sound feedback link is the audio transmission interface of the electronic device (such as a headphone interface or a Bluetooth module), and the data sending port of the ambient sound feedback link is the signal processing module in the digital signal processor.

[0028] In this way, when it is detected that the electronic device has entered pocket mode, it means that the microphone of the electronic device is blocked. At this time, the application processor controls the sound pickup device to switch from the microphone of the electronic device to the microphone of the headset, and drives the microphone of the headset to collect the ambient audio signal in real time; at the same time, since the data acquisition port of the ambient audio signal has changed, the application processor also controls the digital signal processor to create a new ambient sound feedback link, and then obtains the ambient audio signal through the new ambient sound feedback link, so as to achieve the collection of ambient audio signals with the help of the headset microphone when the mobile phone microphone is blocked, thereby improving the accuracy of the acquired ambient audio signal.

[0029] According to the first aspect, or any implementation of the first aspect above, in the system, the electronic device is also used to determine whether the headset has an ambient sound control function based on the device attribute information of the headset after establishing a connection with the headset; wherein the device attribute information of the headset includes a headset device identification; if the headset device identification does not belong to a preset headset whitelist, it is determined that the headset does not have an ambient sound control function.

[0030] In this way, by adding a detection process of whether the headset has the ambient sound control function, only when the headset does not have the ambient sound control function will the relevant processing flow of the ambient sound control on the electronic device side be triggered. If the headset has the ambient sound control function, the relevant processing flow of the ambient sound will not be triggered on the electronic device side, thereby avoiding the ambient sound control on both the electronic device side and the headset side at the same time, which affects the user's audio listening experience.

[0031] In the second aspect, the embodiment of the present application provides an audio processing method. Applied to an electronic device, the electronic device includes a microphone, the control mode of the electronic device for ambient sound includes a noise reduction mode and a transparent mode, and the earphones connected to the electronic device by wire or wirelessly do not have an ambient sound control function. The method includes: obtaining a media audio signal of a preset application; and collecting the ambient audio signal in real time through the microphone; generating a first audio signal according to the current control mode and the ambient audio signal; wherein, when the current control mode is the noise reduction mode, the first audio signal includes a reverse wave audio signal corresponding to the ambient audio signal, and when the current control mode is the transparent mode, the first audio signal includes a forward wave audio signal corresponding to the ambient audio signal; performing signal superposition processing on the first audio signal and the media audio signal to obtain a second audio signal; and sending the second audio signal to the earphone.

[0032] According to the second aspect, in the method, a first audio signal is generated according to a current control mode and an ambient audio signal, including: generating a third audio signal according to the current control mode and the ambient audio signal; wherein, when the current control mode is a noise reduction mode, the third audio signal includes an initial reverse wave audio signal corresponding to the ambient audio signal, and when the current control mode is a transparency mode, the third audio signal includes an initial forward wave audio signal corresponding to the ambient audio signal; according to the noise reduction amplitude of the noise reduction mode, the amplitude of the initial reverse wave audio signal is increased or decreased to obtain a reverse wave audio signal; or, according to the transparency amplitude of the transparency mode, the amplitude of the initial forward wave audio signal is increased or decreased to obtain a forward wave audio signal.

[0033] According to the second aspect, or any implementation of the second aspect above, in the method, a third audio signal is generated according to the current control mode and the ambient audio signal, including: if the current control mode is the noise reduction mode, performing an ambient sound type analysis according to the ambient audio signal to determine the ambient sound type; determining the adjustable parameter information of the preset reverse wave generation algorithm according to the ambient sound type; and using the preset reverse wave generation algorithm to generate an initial reverse wave audio signal according to the adjustable parameter information and the ambient audio signal.

[0034] According to the second aspect, or any implementation of the second aspect above, the method also includes: identifying the scene type of the user's environment; and obtaining target association information corresponding to the scene type; if it is determined according to the target association information that the first mode switching condition is currently met, the current control mode is switched from the noise reduction mode to the transparency mode; if it is determined according to the target association information that the second mode switching condition is currently met, the current control mode is switched from the transparency mode to the noise reduction mode.

[0035] According to the second aspect, or any implementation of the second aspect above, in the method, after identifying the scene type of the user's environment, it also includes: if the current control mode is the transparent mode, determining the valid audio signal and the invalid audio signal in the ambient audio signal according to the scene type; wherein the first audio signal includes a forward wave audio signal corresponding to the valid audio signal and a reverse wave audio signal corresponding to the invalid audio signal.

[0036] According to the second aspect, or any implementation of the second aspect above, in the method, the electronic device also includes an application processor and a digital signal processor; obtaining a media audio signal of a preset application; and collecting an ambient audio signal in real time through a microphone, including: obtaining a media audio signal of a preset application through the application processor, and determining a mode identifier of a current control mode; driving the microphone to collect an ambient audio signal in real time through the application processor; and sending a first indication information and a mode identifier to the digital signal processor; creating an audio playback link and a first feedback link through the digital signal processor according to the first indication information; and obtaining a media audio signal from the application processor through the audio playback link; and obtaining an ambient audio signal from the microphone through the first feedback link. Correspondingly, generating a first audio signal according to the current control mode and the ambient audio signal includes: generating a first audio signal according to the mode identifier and the ambient audio signal through the digital signal processor.

[0037] According to the second aspect, or any implementation of the second aspect above, the method also includes: when it is detected that the electronic device enters pocket mode, sending a microphone turn-on instruction to the headset through the application processor; and sending a second indication information to the digital signal processor; destroying the first feedback link and creating a second feedback link through the digital signal processor according to the second indication information, so as to obtain an ambient audio signal from the audio transmission interface of the electronic device through the second feedback link; the ambient audio signal is an audio signal that the headset controls the microphone of the headset to collect in real time and transmit to the audio transmission interface in response to the microphone turn-on instruction.

[0038] According to the second aspect, or any implementation of the second aspect above, in the method, before collecting the ambient audio signal in real time through the microphone, it also includes: after establishing a connection with the headset, judging whether the headset has the ambient sound control function according to the device attribute information of the headset; wherein the device attribute information of the headset includes the headset device identification; if the headset device identification does not belong to the preset headset whitelist, it is determined that the headset does not have the ambient sound control function.

[0039] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0040] In a third aspect, an embodiment of the present application provides an electronic device. The electronic device includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the audio processing method as described in the second aspect and any one of the second aspects.

[0041] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0042] Exemplarily, the electronic device may be a terminal device or a chip in a terminal device. The electronic device may include an input unit and a processing unit. When the electronic device is a terminal device, the processing unit may be a processor, and the input unit may be a communication interface; the terminal device may also include a memory, the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the terminal device executes any one of the audio processing methods in the second aspect.

[0043] When the electronic device is a chip in a terminal device, the processing unit may be a processing unit inside the chip, and the input unit may be an output interface, a pin or a circuit, etc.; the chip may also include a memory, which may be a memory inside the chip (for example, a register, a cache, etc.) or a memory located outside the chip (for example, a read-only memory, a random access memory, etc.); the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip executes any one of the audio processing methods in the second aspect.

[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium includes a computer program, and when the computer program is executed on an electronic device, the electronic device executes the audio processing method of the second aspect and any one of the second aspects.

[0045] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0046] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed, enables a computer to execute an audio processing method as described in the second aspect or any one of the second aspects.

[0047] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0048] In a sixth aspect, the present application provides a chip, the chip comprising a processing circuit and a transceiver pin, wherein the transceiver pin and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the audio processing method of the second aspect or any one of the second aspects to control the receiving pin to receive a signal, and to control the sending pin to send a signal.

[0049] The sixth aspect and any implementation of the sixth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0050] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it is understood that the description of features or beneficial effects means that specific technical features, technical solutions or beneficial effects are included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the present embodiment can also be combined in any appropriate manner. Those skilled in the art will understand that the embodiment can be realized without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in specific embodiments that do not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1The following is a schematic diagram of an application scenario of an audio processing system shown as an example;

[0052] Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present application is exemplified;

[0053] Figure 3 is a schematic diagram of the hardware structure of an electronic device shown as an example;

[0054] Figure 4 A schematic diagram of the software structure of an electronic device is shown as an example;

[0055] Figure 5 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0056] Figure 6 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0057] Figure 7 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0058] Figure 8 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0059] Fig. 9 A schematic diagram of an electronic device framework provided in an embodiment of the present application;

[0060] Fig.10 A schematic diagram of an electronic device framework provided in an embodiment of the present application;

[0061] Fig.11 A schematic diagram of an electronic device framework provided in an embodiment of the present application;

[0062] Fig.12 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0063] Fig.13 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0064] Fig.14 A schematic diagram of an application scenario provided for an embodiment of the present application. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0067] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0068] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0069] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0070] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0071] To facilitate understanding, relevant terms and concepts involved in some embodiments provided in this application are first introduced below.

[0072] Active Noise Control (ANC) is a technology that reduces ambient noise by introducing noise that is opposite to the input signal.

[0073] The control mode of ambient sound may be referred to as an ambient sound control mode, and the ambient sound control mode may include a noise reduction mode and a transparency mode.

[0074] The noise reduction mode is to collect ambient noise in real time and generate a reverse sound wave signal of the ambient noise, so that the reverse sound wave signal and the sound wave signal of the ambient noise entering the user's ear are superimposed and offset to achieve the effect of noise reduction, so that the user can enjoy a quiet listening experience in a noisy environment. In addition, in the noise reduction mode, media audio can be understood as the audio that the user wants to hear, and ambient audio can be understood as the audio that the user wants to block.

[0075] Transparency mode refers to the real-time collection of ambient noise and the generation of a positive sound wave signal of the ambient noise, so that the positive sound wave signal is superimposed and enhanced with the sound wave signal of the ambient noise entering the user's ear to achieve a transparent effect. In this way, the user can clearly perceive the sounds of the surrounding environment while wearing headphones, such as the sound of vehicles driving, human voices, site announcements, etc.

[0076] The strength adjustment amplitude of the ambient sound control effect is used to adjust the fluctuation amplitude of the target ambient audio signal in the process of generating the target ambient audio signal, so as to adjust the noise reduction strength or transparency strength perceived by the user.

[0077] Ambient audio refers to the environmental noise that directly enters the user's ears when the user uses a mobile phone or other electronic device to listen to audio. Ambient noise can include the sound of driving vehicles, human voices, construction site sounds, radio broadcasts, etc. All audio except the media audio played by the mobile phone can be called ambient audio.

[0078] The ambient audio signal is an audio signal obtained by converting the sound wave signal of the ambient audio collected in real time by the microphone (i.e., converting the sound wave signal into an electrical signal). The audio signal can be an audio analog electrical signal or an audio digital electrical signal. The ambient audio signal reflects the waveform information of the sound wave signal of the ambient audio, such as the amplitude, frequency and phase of the sound wave signal.

[0079] The reverse wave audio signal corresponding to the ambient audio signal is an audio signal obtained by performing phase inversion processing on the ambient audio signal using a preset reverse wave generation algorithm, and the obtained audio signal can be called the reverse wave audio signal of the ambient audio. The reverse wave audio signal and the ambient audio signal have opposite phases in the frequency domain. Therefore, the sound wave signal corresponding to the reverse wave audio signal is opposite in phase to the sound wave signal of the ambient audio that directly enters the user's ear. Among them, the sound wave signal corresponding to the reverse wave audio signal refers to the sound wave signal obtained by performing signal conversion on the reverse wave audio signal of the ambient audio (i.e., converting the audio signal into a sound wave signal), which can be called the reverse sound wave signal of the ambient audio.

[0080] The positive wave audio signal corresponding to the ambient audio signal can be the ambient audio signal, or it can be an audio signal obtained by performing preset processing (such as gain, clipping, etc.) on the ambient audio signal, which can be called the positive wave audio signal of the ambient audio. The positive wave audio signal and the ambient audio signal have the same phase in the frequency domain. Therefore, the sound wave signal corresponding to the positive wave audio signal has the same phase as the sound wave signal of the ambient audio that directly enters the user's ear. Among them, the sound wave signal corresponding to the positive wave audio signal refers to the sound wave signal obtained by performing signal conversion on the positive wave audio signal of the ambient audio (that is, converting the audio signal into a sound wave signal), which can be called the positive sound wave signal of the ambient audio.

[0081] It can be understood that the signals involved in the processing and transmission of ambient audio are called audio signals, and the process of ambient audio propagating in the air is called sound wave signals.

[0082] Next, the specific implementation process of some embodiments provided in this application is described in detail.

[0083] It is understandable that after a wired or wireless connection is established between an electronic device (such as a mobile phone, tablet computer, etc.) and a headset, the user can make calls through the headset or listen to the audio played by the mobile phone through the headset. In specific implementation, if the user is in a noisy on-site environment, in order to improve the call quality during the call and enable the other party to clearly hear the speaker's voice, environmental noise cancellation technology (ENC) is usually used on the mobile phone side to identify and filter out various types of interfering noises. Such as the conversations of the surrounding people, vehicle noise, etc., to effectively suppress the background noise in the environment, thereby reducing the impact of environmental noise on call quality and improving the clarity of the call.

[0084] However, ENC technology is mainly used to perform noise reduction processing on audio data that needs to be transmitted to the other party of the call, so that the other party of the call can hear the user's voice clearly and purely. For application scenarios where users use headphones to listen to audio played by mobile phones, it is necessary to enable the local user wearing headphones to enjoy a quiet listening environment in a noisy on-site environment. Therefore, users can wear high-end headphones with noise reduction function. Because the headphone side uses active noise reduction technology (Active Noise Cancellation, ANC), the external environmental noise is collected through the microphone, and then converted into an anti-phase sound wave of the environmental noise through algorithm processing and added to the headphone speaker, and then the environmental noise directly entering the user's ear is offset by the principle of destructive interference. Figure 1 FIG. 1 is a schematic diagram showing an example of an application scenario of an audio processing system. Figure 1For example, taking the case where the earphone 200 is a Bluetooth earphone with a noise reduction function, when the mobile phone 101 and the earphone 200 have not established a Bluetooth connection, the mobile phone 101 and the earphone 200 can respectively respond to the user's Bluetooth operation to turn on the Bluetooth function. Then, the mobile phone 101 establishes a Bluetooth connection with the earphone 200 in response to the user's Bluetooth pairing operation. After the mobile phone 101 and the earphone 200 establish a Bluetooth connection for the first time, as long as the mobile phone 101 and the earphone 200 both turn on the Bluetooth function and the distance between the two is less than a preset threshold, the mobile phone 101 and the earphone 200 can automatically establish a Bluetooth connection. After the mobile phone 101 and the earphone 200 establish a Bluetooth connection, the earphone 200 implements the noise reduction mode function and the transparent mode function through the microphone of the earphone and the specific noise reduction processing chip in the earphone.

[0085] Among them, when the earphone 200 turns on the noise reduction mode, the earphone 200 receives the media audio signal transmitted by the mobile phone 101, and uses the microphone of the earphone to collect the sound wave signal of the ambient audio in the surrounding environment, converts the sound wave signal into the ambient audio signal, and uses the specific noise reduction processing chip in the earphone to generate the ambient audio reverse wave signal according to the ambient audio signal. Then, the speaker of the earphone is used to play the sound wave signal corresponding to the ambient audio reverse wave signal (referred to as the reverse sound wave signal of the ambient audio) and the sound wave signal corresponding to the media audio signal transmitted by the mobile phone 101. At this time, the reverse sound wave signal of the ambient audio heard by the ears of the local user wearing the earphone 200 is opposite to the phase of the forward sound wave signal of the ambient audio that directly enters the user's ears. At this time, the ambient sound can be offset, so that the audio actually heard by the user is the media audio.

[0086] In addition, it can be understood that when the headset 200 turns on the transparent mode, the headset 200 receives the media audio signal transmitted by the mobile phone 101, and uses the microphone of the headset to collect the sound wave signal of the ambient audio in the surrounding environment, converts the sound wave signal into the ambient audio signal, and uses the specific noise reduction processing chip in the headset to generate the ambient audio positive wave signal according to the ambient audio signal. Then, the speaker of the headset 200 plays the sound wave signal corresponding to the ambient audio positive wave signal (referred to as the positive sound wave signal of the ambient audio) and the sound wave signal corresponding to the media audio signal transmitted by the mobile phone 101. At this time, the positive sound wave signal of the ambient audio heard by the ear of the local user wearing the headset 200 is the same as the phase of the positive sound wave signal of the ambient audio directly entering the user's ear. At this time, the ambient sound can be enhanced, so that the user can clearly hear the surrounding environment sound when wearing the headset 200.

[0087] It can be seen that the purpose of creating a quieter listening environment for users is to achieve active noise reduction through headphones. The more additional functions headphones have, the more expensive they are. In other words, users must buy high-end headphones with these functions to enjoy noise reduction or transparent experience. Therefore, for users who use ordinary headphones, since the headphones do not have noise reduction function, users cannot enjoy a quiet listening environment in noisy live environments.

[0088] In order to solve the above problems, an embodiment of the present application provides an audio playback method. In this method, an active noise reduction function is implemented by an electronic device (such as a mobile phone, a tablet computer, etc.). Figure 2 To exemplify an application scenario provided by an embodiment of the present application, refer to Figure 2 , for example, taking the case where the earphone 200 can be a Bluetooth earphone without a noise reduction function, for the scenario where the user opens the music application 204 to listen to music, when the switch control of the noise reduction mode is in the on state, it means that the ambient sound control mode of the mobile phone is set to the noise reduction mode. The processor 110 in the mobile phone not only needs to obtain the media audio signal of the music application 204, but also needs to obtain the ambient audio signal collected by the microphone 170C (that is, the microphone 170C converts the sound wave signal of the ambient audio to obtain the audio signal). Then, the processor 110 generates a reverse wave audio signal of the ambient audio based on the ambient audio signal; then the reverse wave audio signal is processed by signal superposition with the media audio signal to obtain a target superposition audio signal. The target superposition audio signal is then transmitted to the earphone 200 using the Bluetooth module 202, and the speaker of the earphone 200 converts the target superposition audio signal into a target superposition sound wave signal, and plays the target superposition sound wave signal.

[0089] In this way, when the ambient sound control mode of the mobile phone is set to the noise reduction mode, since the target superimposed audio signal sent by the mobile phone to the earphone end includes not only the media audio signal, but also the reverse wave audio signal of the ambient audio, the sound wave signal corresponding to the reverse wave audio signal is opposite in phase to the forward sound wave signal of the ambient audio that directly enters the user's ear, the ambient audio can be offset at this time, so that the audio that the user actually hears is the audio of the song currently playing in the music application. In other words, by collecting ambient audio signals with the help of an idle microphone in the mobile phone (the microphone is in an idle state when audio is playing), and with the help of the audio signal processing capability of the processor in the mobile phone, the active noise reduction function is realized on the mobile phone side, so that even if the headphones worn by the user do not have a noise reduction function, a quiet listening environment can be provided for the user.

[0090] Moreover, the ambient sound control mode of the mobile phone includes not only the noise reduction mode, but also the transparency mode. Therefore, when the ambient sound control mode of the mobile phone is set to the transparency mode, the target superimposed audio signal includes not only the media audio signal, but also the forward wave audio signal of the ambient audio; the sound wave signal corresponding to the forward wave audio signal has the same phase as the forward sound wave signal of the ambient audio that directly enters the user's ear, and the ambient audio can be enhanced at this time, so that the user can clearly hear the ambient sound when wearing headphones, and the user does not need to repeatedly take off or put on the headphones, so that the user can switch between shielding the ambient sound and enhancing the ambient sound at will in terms of hearing.

[0091] In addition, the mobile phone does not directly send the media audio signal and the target ambient audio signal (the reverse wave audio signal or the forward wave audio signal of the ambient audio) to the earphone separately, but sends the target superimposed audio signal after the two are superimposed to the earphone. This reduces the hardware requirements for the earphone, and the earphone does not need to have the ability to process two audio streams at the same time, thereby improving the adaptability of the earphone.

[0092] First, the hardware structure of the electronic device 100 is schematically introduced in detail.

[0093] Figure 3 The hardware structure diagram of the electronic device 100 is exemplarily shown. It should be understood that Figure 3 The illustrated electronic device 100 is merely one example of an electronic device, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 3 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0094] like Figure 3 As shown, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, a first antenna, a second antenna, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0095] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it may also process other digital signals.

[0096] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0097] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0098] The wireless communication function of the electronic device 100 can be implemented by the first antenna, the second antenna, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor and the baseband processor. The first antenna and the second antenna are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna. For example, the first antenna can be reused as a diversity antenna of a wireless local area network.

[0099] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The wireless communication module 160 can provide solutions for wireless communications including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., applied to the electronic device 100. In some embodiments, the first antenna of the electronic device 100 is coupled to the mobile communication module 150, and the second antenna is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0100] In some embodiments of the present application, as an example, in the case where the earphones worn by the user are Bluetooth earphones, Bluetooth (BT) is used to establish a Bluetooth connection with the Bluetooth earphones, and the above-mentioned target superimposed audio signal is transmitted to the Bluetooth earphones via Bluetooth.

[0101] The electronic device 100 implements the display function through the GPU, the display screen 194, and the application processor, etc. In some embodiments of the present application, as an example, the display screen 194 is also used to display the user interaction interface of the preset application and the setting interface of the ambient sound control mode, so that the user can manually set the ambient sound control mode.

[0102] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0103] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0104] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0105] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0106] Among them, the speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or voice message, the voice can be answered by placing the receiver 170B close to the human ear. The microphone 170C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching the microphone 170C with the human mouth to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals, but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc. The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130, or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0107] In some embodiments of the present application, as an example, microphone 170C is used to collect sound wave signals of ambient audio and convert the sound wave signals of ambient audio into digital signals (which may be called ambient audio signals), and the digital signal processor obtains the ambient audio signals from microphone 170C; in the case where the user wears wired headphones, headphone interface 170D is used to establish a communication connection with the wired headphones, and the above-mentioned target superimposed audio signal is transmitted to the wired headphones through headphone interface 170D.

[0108] The sensor module 180 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc., which are not listed here one by one and the present application does not impose any limitation on this.

[0109] Next, the software structure of the electronic device 100 is schematically introduced and described in detail.

[0110] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100. Figure 4 A schematic diagram of the software structure of the electronic device 100 according to an embodiment of the present application is shown.

[0111] The layered architecture of the electronic device 100 divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime (Android Runtime) and the system library, and the kernel layer. In practical applications, the above-mentioned layered architecture can also include a hardware abstraction layer, which is located between the system library and the kernel layer.

[0112] The application layer can include a series of application packages. Figure 4 As shown, the application package may include applications such as camera, calendar, map, gallery, call, navigation, WLAN, etc. The application package may also include applications such as music, video, settings, Bluetooth, etc.

[0113] In some embodiments of the present application, as an example, the application layer may further include a perception module, which is used to perceive the scene type of the user's environment, for example, the scene type may include an office scene, a scene of riding a vehicle, etc. The perception module is also used to perceive whether the electronic device enters the pocket mode.

[0114] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0115] like Figure 4 As shown, the application framework layer may include a window manager, a content provider, a view system, a notification manager, and the like.

[0116] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0117] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0118] The view system can include visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0119] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0120] In some embodiments of the present application, as an example, the application framework layer may also include an audio stream control service, a configuration detection service, and a mode control service. Among them, the audio stream control service is used to control the transmission process of the audio stream to the digital signal processor; the configuration detection service is used to detect whether the earphone connected to the electronic device has an ambient sound control function; the mode control service is used to send an ambient sound control mode identifier to the hardware abstraction layer when the earphone does not have an ambient sound control function, so as to control the digital signal processor to create an audio playback link and an ambient sound feedback link.

[0121] Android Runtime includes core libraries and virtual machines. Android Runtime is responsible for scheduling and management of the Android system. The core library consists of two parts: one is the function that the Java language needs to call, and the other is the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0122] The system library can include multiple functional modules. Figure 4As shown, for example: surface manager, media library, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc. The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0123] The hardware abstraction layer, also known as the HAL layer, provides a unified interface for upper-layer applications to access hardware devices. Figure 4 As shown, the hardware abstraction layer includes but is not limited to: Camera Hardware Abstraction Layer (CameraHAL). Among them, Camera HAL is used to process the image stream.

[0124] In some embodiments of the present application, as an example, the hardware abstraction layer may also include an audio hardware abstraction layer, commonly referred to as an audio HAL, which is used to drive the microphone to power on through an audio driver when the headset does not have an ambient sound control function and the ambient sound control mode identifier includes a noise reduction mode or a transparent mode, so that the microphone can collect ambient audio signals. In addition, the access address of the media audio signal and the ambient sound control mode identifier are sent to the digital signal processor through the inter-core communication interface, and the creation of the audio playback link and the ambient sound feedback link is controlled based on the ambient sound control mode identifier, so that the digital signal processor obtains the media audio signal from the application framework layer, and obtains the ambient audio signal from the microphone, and then generates a target superimposed audio signal based on the media audio signal and the ambient audio signal. Exemplarily, the target ambient audio signal is first generated based on the ambient sound control mode and the ambient audio signal, and then the target ambient audio signal is superimposed with the media audio signal to obtain the target superimposed audio signal.

[0125] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, audio driver, headphone jack driver, Bluetooth module driver, Wi-Fi driver, camera driver, sensor driver, etc. Among them, the audio driver may include microphone driver, and the hardware includes at least processor, microphone, Bluetooth module, headphone jack, display, Wi-Fi module, sensor, etc.

[0126] Understandably, Figure 4The layers in the software structure shown and the components included in each layer are only examples and do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown in the figure, and each layer may include more or fewer components, or combine some components, or split some components, or arrange the components differently, which is not limited in the present application.

[0127] The embodiment of the present application provides an audio playback system. Specifically, the ambient sound control function is implemented on the electronic device side. If the ambient sound control mode of the electronic device is a noise reduction mode, the user can enjoy a quiet listening environment at this time; if the ambient sound control mode of the electronic device is a transparent mode, the user can also clearly hear the surrounding ambient sound when wearing headphones. In this way, the user does not have to wear high-end headphones with an ambient sound control function, so that even if the user only wears ordinary headphones without an ambient sound control function, he can enjoy a quiet listening environment (which can be understood as shielding the ambient sound) or clearly hear the surrounding ambient sound (which can be understood as shielding the ambient sound). In addition, since the electronic device side has not only a noise reduction mode but also a transparent mode, the user does not need to repeatedly take off or put on the headphones, and can switch between shielding the ambient sound and enhancing the ambient sound at will in terms of hearing. The following explanation takes a mobile phone as an example, and the embodiments of the present application will not be repeated with regard to other types of electronic devices.

[0128] Next, the specific implementation process of the audio processing method provided in the embodiment of the present application is explained in combination with a specific application scenario example.

[0129] The following is a scenario in which a user uses headphones to listen to songs played by a mobile phone, and the specific implementation process of the electronic device performing audio processing will be described in detail with reference to the accompanying drawings.

[0130] For ease of understanding, before the audio processing system provided in the embodiment of the present application is introduced in detail, the setting process of the ambient sound control mode involved in the embodiment of the present application is described. Figures 5 to 8 The following is a schematic diagram of an application scenario, specifically a schematic diagram of the setting operation interface of the ambient sound control mode. In this example, the electronic device takes a mobile phone as an example. If the ambient sound control function is implemented on the mobile phone side, the setting interface of the ambient sound control mode is provided to the user on the mobile phone side. Among them, as for the time node of setting the ambient sound control mode, the ambient sound control mode can be set before using the mobile phone to play audio, or it can be set during the process of using the mobile phone to play audio, or it can be set after using the mobile phone to play audio.

[0131] As for the specific implementation method of the setting process of the ambient sound control mode, the user can manually set the ambient sound control mode; or the electronic device automatically sets the ambient sound control mode based on the intelligent analysis results; or the electronic device automatically provides a mode setting prompt based on the intelligent analysis results to prompt the user to manually set the ambient sound control mode.

[0132] Optionally, take the example of a user manually setting the ambient sound control mode. For example, the user opens a setting application and enters a user operation interface for setting the ambient sound control mode to set it. For another example, the user enters a notification bar interface to set the ambient sound control mode. For another example, the user enters a control center interface to set the ambient sound control mode.

[0133] In addition to setting the ambient sound control mode, you can also use the designated adjustment controls to adjust the strength of the ambient sound control effect.

[0134] like Figure 5 As shown, use the Settings app to set the ambient sound control mode, see Figure 5 In (a), the mobile phone desktop is displayed, and the user clicks the application icon 501 of the setting application; the mobile phone responds to the user operation and displays the following Figure 5 In the user operation interface of the setting application shown in (b) in FIG. 1 , the user clicks the “ambient sound control mode” setting control 502; the mobile phone responds to the user operation and displays the following Figure 5 The setting interface of the ambient sound control mode shown in (c) of FIG. 1 includes a first switch control 503 (i.e., a control for setting the switch state of the noise reduction mode, which can be referred to as the switch control of the noise reduction mode). The user clicks the first switch control 503; in response to the user operation, see Figure 5 In (d), the first switch control 503 is in the on state, and the ambient sound control mode is the noise reduction mode.

[0135] like Figure 6 As shown, for the process in which the user uses the specified adjustment control to adjust the strength of the ambient sound control effect, take the adjustment process of the noise reduction effect as an example, see Figure 6 In (a), the mobile phone displays the setting interface of the ambient sound control mode, and the user clicks the "strength adjustment bar" display control 504; the mobile phone responds to the user operation and displays the following Figure 6 The noise reduction strength adjustment bar 505 shown in (b) of FIG. 1 is a display of the mobile phone. The user drags the first movable member 506 on the noise reduction strength adjustment bar 505, and the mobile phone responds to the user operation and displays the following Figure 6The first movable member 506 shown in (c) moves from the first position (such as the 50% position) of the noise reduction strength adjustment bar 505 to the second position (such as the 70% position), and the strength adjustment range in the noise reduction mode is set from 50% to 70%.

[0136] like Figure 7 As shown, the ambient sound control mode can be set through the notification bar interface, see Figure 7 In (a), the mobile phone desktop is displayed. The user pulls down from the top center of the mobile phone. In response to the user operation, the mobile phone displays the following Figure 7 The display interface of the notification bar shown in (b) in FIG. 1 includes a notification about the noise reduction mode setting. The user clicks the first switch control 503. In response to the user operation, see Figure 7 In (c), the first switch control 503 is in the on state, and the ambient sound control mode is the noise reduction mode. In addition, the strength of the noise reduction effect can be adjusted by the first movable member 506 on the noise reduction strength adjustment bar 505.

[0137] like Figure 8 As shown, set the ambient sound control mode through the control center interface, see Figure 8 In (a), the mobile phone desktop is displayed. The user pulls down from the top right of the mobile phone. In response to the user operation, the mobile phone displays the following Figure 8 The display interface of the control center shown in (b) in FIG. 1 shows a first switch control 503, a noise reduction strength adjustment bar 505, a first movable member 506, and related setting controls for the transparency mode. The user clicks the first switch control 503; the mobile phone responds to the user operation, see Figure 8 In (c), the first switch control 503 is in the on state, and the ambient sound control mode is the noise reduction mode. In addition, the strength of the noise reduction effect can be adjusted by the first movable member 506 on the noise reduction strength adjustment bar 505.

[0138] Scene 1

[0139] In this scenario, the earphones connected to the electronic device do not have an ambient sound control function, and the ambient sound control mode of the electronic device is set to a noise reduction mode or a transparency mode.

[0140] Fig. 9The figure is a schematic diagram of an electronic device framework for executing an audio processing method. As an example, the electronic device takes a mobile phone as an example, the earphone takes a Bluetooth earphone as an example (the audio transmission interface is a Bluetooth module), and the media audio signal is a song audio stream in the form of a digital signal as an example for explanation. Among them, the mobile phone may include an application processor, a digital signal processor, a microphone and a Bluetooth module, and the software architecture involved in the application processor may include an application layer, an application framework layer, a hardware abstraction layer and a kernel layer. The application layer may include music applications and setting applications, the application framework layer may include audio stream control services, configuration detection services and mode control services, the hardware abstraction layer may include an audio HAL, and the kernel layer may include an audio driver and a Bluetooth module driver. Fig. 9 As shown in the figure, the main implementation process includes:

[0141] S601, the music application sends the audio stream of the song to be played to the audio stream control service.

[0142] Exemplarily, a user opens a music application, selects a song to be listened to, and clicks a song play control; in response to the user operation, the song audio file of the currently playing song is obtained; the song audio file is parsed to obtain the song audio stream to be played. The parsed song audio file is sent to the application framework layer in the form of an audio stream. In some example embodiments, the song audio stream to be played is sent to the application framework layer at a preset time interval.

[0143] S602: The configuration application sends an ambient sound control mode identifier to the configuration detection service.

[0144] Exemplarily, the ambient sound control mode identifier refers to the mode identifier of the current control mode, which is the identifier of the noise reduction mode or the identifier of the transparency mode. If the ambient sound control mode identifier includes the identifier of the noise reduction mode, it means that the switch control of the noise reduction mode is in the on state and the switch control of the transparency mode is in the off state, and the ambient sound control mode is set to the noise reduction mode. If the ambient sound control mode identifier includes the identifier of the transparency mode, it means that the switch control of the transparency mode is in the on state and the switch control of the noise reduction mode is in the off state, and the ambient sound control mode is set to the transparency mode.

[0145] The mode setting information of the ambient sound control mode can be referred to the above Figure 5 , Figure 7 , Figure 8During the song playing process, the mode setting information may be changed; for example, the ambient sound control mode is switched under the user's trigger operation, or the electronic device automatically switches the ambient sound control mode when the preset mode switching condition is met, or the electronic device reminds the user to switch the ambient sound control mode when the preset mode switching condition is met.

[0146] It is understandable that during the song playing process, the application layer will continuously send the song audio stream to the application framework layer. The triggering condition for transmitting the ambient sound control mode identifier from the application layer to the application framework layer can be any of the following: when the song starts playing, the ambient sound control mode identifier changes, or the preset mode identifier sending condition is met.

[0147] That is to say, from the beginning to the end of song playback, it is not necessary to send the ambient sound control mode identifier every time the song audio stream is sent to the application framework layer. If the ambient sound control mode identifier has not changed, the ambient sound control mode identifier is only sent to the application layer once. That is, the above step S601 is executed multiple times, while the above step S602 is only executed once. If the ambient sound control mode identifier changes, the next time the song audio stream is sent to the application layer, the changed ambient sound control mode identifier is sent to the application framework layer.

[0148] S603: The configuration detection service sends an ambient sound control mode identifier to the mode control service.

[0149] Exemplarily, after receiving the ambient sound control mode identifier, first determine whether the headset connected to the mobile phone has the ambient sound control function. If the headset does not have the ambient sound control function, send the ambient sound control mode identifier to the mode control service. If the headset has the ambient sound control function, send indication information for triggering the mobile phone to stop performing ambient sound control to the mode control service; for example, send single link creation indication information to the mode control service.

[0150] For example, after the mobile phone establishes a Bluetooth connection with the headset, the application framework layer obtains the device attribute information of the headset through the hardware abstraction layer and the kernel layer, and determines whether the headset has the ambient sound control function based on the device attribute information. If the headset does not have the ambient sound control function, the relevant processing flow of executing the ambient sound control on the mobile phone side is triggered.

[0151] In some example embodiments, the device attribute information may include a headset device identifier. The electronic device pre-stores a headset whitelist, and the headset whitelist includes a headset device identifier that does not have an ambient sound control function. Then, if the headset device identifier belongs to the headset whitelist, it is determined that the headset does not have an ambient sound control function. The headset whitelist may be updated based on the product introduction information of the headset, or may be updated based on the user's confirmation feedback information that the headset does not have an ambient sound control function.

[0152] It is understandable that if it is detected that a new headset is connected to the mobile phone, it is necessary to determine again through the configuration detection service in the application framework layer based on the device attribute information of the newly connected headset to determine whether the headset does not have the ambient sound control function, and then decide whether to trigger the relevant processing flow of executing the ambient sound control on the mobile phone side when sending the next frame of song audio stream to the hardware abstraction layer.

[0153] S604: The mode control service sends an ambient sound control mode identifier to the audio HAL.

[0154] Exemplarily, when the currently connected headset does not have the ambient sound control function, the configuration detection service sends the ambient sound control mode identifier to the mode control service, and then the mode control service sends the ambient sound control mode identifier to the audio HAL. In other words, when the headset does not have the ambient sound control function, the application framework layer not only needs to send the access address of the song audio stream to the hardware abstraction layer, but also needs to send the ambient sound control mode identifier to the hardware abstraction layer.

[0155] S605: The audio stream control service sends the access address of the song audio stream to the audio HAL.

[0156] For example, after receiving the song audio stream sent by the music application, the song audio stream is first stored in a designated storage location, and the corresponding access address of the song audio stream is generated. In this way, it is only necessary to send the access address of the song audio stream to the hardware abstraction layer and the digital signal processor in turn, and the digital signal processor can directly obtain the song audio stream based on the access address.

[0157] S606, the audio HAL sends a media audio acquisition instruction to the audio playback link to trigger the audio playback link to acquire the song audio stream from the application framework layer.

[0158] Exemplarily, the media audio acquisition instruction may carry the access address of the song audio stream. Since the audio signal processing process is performed by the digital signal processor, the application framework layer does not directly transmit the song audio stream to the hardware abstraction layer, but transmits the access address of the song audio stream, and then the hardware abstraction layer transmits the access address to the digital signal processor, so that the digital signal processor can directly obtain the corresponding song audio stream based on the access address request.

[0159] S607: The audio HAL sends an ambient audio acquisition instruction to the ambient sound feedback link to trigger the ambient sound feedback link to acquire the ambient audio signal from the microphone of the mobile phone.

[0160] The ambient audio signal is an audio signal obtained by converting the sound wave signal of the ambient audio collected in real time by the microphone. Since the audio signal processing process is performed by the digital signal processor, and when the song starts playing, the hardware abstraction layer has triggered the digital signal processor to create an ambient sound feedback link, the process of obtaining the ambient audio signal does not need to go through the hardware abstraction layer and the application layer program framework layer, but directly obtains the ambient audio signal from the microphone of the mobile phone through the ambient sound feedback link.

[0161] Exemplarily, in the case where the earphones do not have an ambient sound control function, the audio playback link and the ambient sound feedback link may be created by a digital signal processor when a song starts playing, and the microphone power-on process may be driven by an application processor when a song starts playing.

[0162] When the song starts playing, if the ambient sound control mode identifier received by the audio HAL includes the identifier of the noise reduction mode or the identifier of the transparent mode, it means that the digital signal processor needs to be controlled to obtain the media audio signal and the ambient audio signal. Therefore, it is necessary to trigger the digital signal processor to create an audio playback link and an ambient sound feedback link. The audio playback link is used to obtain the song audio stream from the application framework layer and transmit the song audio stream to the digital signal processor; the ambient sound feedback link is used to obtain the ambient audio signal from the microphone of the mobile phone and transmit the ambient audio signal to the digital signal processor. In some example embodiments, the audio HAL sends a dual-link creation instruction to the digital signal processor; wherein the dual-link creation instruction includes an audio playback link creation instruction and an ambient sound feedback link creation instruction.

[0163] Exemplarily, the process of creating an audio playback link mainly includes creating an audio stream acquisition port and an audio stream transmission port. The audio stream acquisition port is used to indicate where to obtain the song audio stream, and the audio stream transmission port is used to indicate which module to send the song audio stream to. The process of creating an ambient sound feedback link mainly includes creating an ambient audio acquisition port and an ambient sound transmission port. The ambient audio acquisition port is used to indicate where to obtain the ambient audio signal, and the ambient sound transmission port is used to indicate which module to send the ambient audio signal to.

[0164] It can be understood that when the song starts playing, the audio HAL sends a dual-link creation instruction to the digital signal processor to control the digital signal processor to create the required audio playback link and ambient sound feedback link; then, during the song playback process, if the connection of headphones with ambient sound control function is not detected, the audio playback link and the ambient sound feedback link will not be destroyed, and there is no need to trigger the creation of the audio playback link and the ambient sound feedback link again.

[0165] In addition, since the ambient audio signal needs to be collected in real time, the audio HAL also needs to send a microphone drive instruction to the kernel layer. After receiving the microphone drive instruction, the kernel layer triggers the microphone to power on. In response to the power-on drive instruction of the kernel layer, the microphone powers on and collects the ambient audio signal.

[0166] It is understandable that when the song starts playing, the hardware abstraction layer sends a power-on drive instruction to the microphone through the kernel layer to trigger the microphone to power on and be in the on state; then, during the song playback, if the headset with ambient sound control function is not detected, the microphone remains in the on state without triggering the microphone to power on again. In addition, the power-on process of the Bluetooth module can be performed when the mobile phone and the headset are connected by Bluetooth. If the trigger condition of Bluetooth disconnection is not detected, the Bluetooth module can also remain in the on state.

[0167] Furthermore, considering that the mobile phone microphone may be blocked, it is bound to affect the accuracy of the ambient audio signal collected by the mobile phone microphone. In order to improve the accuracy of the collection of the ambient audio signal, the accuracy of the generation of the target ambient audio signal is improved. Therefore, with the help of the pocket mode detection function of the mobile phone, when the mobile phone is detected to enter the pocket mode, it means that the mobile phone microphone may be blocked. At this time, the perception module in the application layer sends the microphone blocking prompt information to the application framework layer; after receiving the prompt information, the application framework layer sends the pickup device switching instruction to the hardware abstraction layer; the hardware abstraction layer responds to the pickup device switching instruction, switches the pickup device from the microphone of the mobile phone to the microphone of the headset, so as to control the microphone of the headset to collect the ambient audio signal in real time, and transmit the ambient audio signal to the audio transmission interface of the electronic device; and, since the data acquisition port of the ambient audio signal has changed, the hardware abstraction layer also needs to send the new indication information of the ambient sound feedback link to the digital signal processor to trigger the digital signal processor to create a new ambient sound feedback link. Then, the ambient audio signal is obtained from the audio transmission interface through the newly created ambient sound feedback link. This makes it possible to collect ambient audio signals with the help of the headphone microphone when the mobile phone microphone is blocked, thereby improving the accuracy of the acquired ambient audio signals.

[0168] Exemplarily, pocket mode may refer to a user placing a mobile phone in a trouser pocket or a backpack while walking. For another example, a user may place a mobile phone in a drawer, etc. The detection process of the pocket mode of the mobile phone may collect the ambient light intensity through the ambient light sensor of the mobile phone; based on the ambient light intensity and the current time information, it may be determined whether to enter the pocket mode. The distance sensor of the mobile phone may also be used to detect the distance between the object and the mobile phone screen; based on the distance, it may be determined whether to enter the pocket mode. It is also possible to combine the detection results of the ambient light sensor and the distance sensor to determine whether to enter the pocket mode. The specific implementation method may be set according to actual needs, and this application does not limit this.

[0169] S608: The audio HAL sends an ambient sound control mode identifier to the signal processing module.

[0170] For example, when the headset does not have an ambient sound control function, the application layer not only needs to send the song audio stream to the application framework layer, but also needs to send the ambient sound control mode identifier to the application framework layer. The application framework layer sends the ambient sound control mode identifier to the audio HAL. The audio HAL sends the ambient sound control mode identifier to the digital signal processor. In this way, the digital signal processor decides whether to generate a reverse wave audio signal of the ambient audio or a forward wave audio signal according to the ambient sound control mode identifier, thereby achieving the effect of shielding the ambient sound or enhancing the ambient sound.

[0171] S609, the audio playback link transmits the song audio stream to the signal processing module.

[0172] S610: The ambient sound feedback link transmits the ambient audio signal to the signal processing module.

[0173] Among them, after receiving the ambient sound control mode identifier, the song audio stream and the ambient sound signal, the signal processing module first generates a target ambient audio signal according to the ambient sound control mode identifier and the ambient audio signal; then superimposes the target ambient audio signal with the song audio stream to obtain a target superimposed audio signal.

[0174] Exemplarily, when the ambient sound control mode is set to the noise reduction mode, the ambient audio signal is first processed using a preset reverse wave generation algorithm to generate a reverse wave audio signal; the reverse wave audio signal is then superimposed with the song audio stream to obtain a first superimposed audio signal; when the ambient sound control mode is set to the transparency mode, a forward wave audio signal corresponding to the ambient audio signal is first generated; the forward wave audio signal is then superimposed with the song audio stream to obtain a second superimposed audio signal; wherein the preset reverse wave generation algorithm includes any one of an adaptive filtering algorithm and a frequency domain filtering algorithm.

[0175] In some example embodiments, during the generation of the reverse wave audio signal of the ambient audio, the digital signal processor may directly use a preset reverse wave generation algorithm to perform signal processing on the ambient audio signal to generate the reverse wave audio signal, wherein the parameter values ​​of the preset reverse wave generation algorithm are fixed and unchanged.

[0176] In some other example embodiments, in the process of generating the reverse wave audio signal, the signal processing module first analyzes the ambient sound type based on the ambient audio signal to determine the ambient sound type; then determines the adjustable parameter information of the preset reverse wave generation algorithm based on the ambient sound type; and generates the reverse wave audio signal of the ambient audio based on the adjustable parameter information and the ambient audio signal using the preset reverse wave generation algorithm. This can improve the accuracy of the generation of the reverse wave audio signal. Among them, considering that the accuracy of distinguishing the frequency components of the ambient audio signal in the frequency domain is higher, the ambient audio signal is first Fourier transformed to obtain the frequency domain signal of the ambient audio, that is, the ambient audio signal in the time domain is converted into the ambient audio signal in the frequency domain; then, the preset reverse wave generation algorithm is used to filter and phase-invert the frequency domain signal of the ambient audio based on the weight coefficients corresponding to each frequency band to obtain the reverse wave frequency domain signal; then, the reverse wave frequency domain signal is inversely Fourier transformed to obtain the reverse wave audio signal of the ambient audio, that is, the reverse wave audio signal in the frequency domain is converted into the reverse wave audio signal in the time domain.

[0177] Exemplarily, the preset reverse wave generation algorithm can be expressed as ,in, represents the high frequency band weight coefficient, The inverse wave generation algorithm for the frequency domain signal corresponding to the high frequency band, represents the mid-frequency band weight coefficient, The inverse wave generation algorithm for the frequency domain signal corresponding to the intermediate frequency band, represents the low frequency band weight coefficient, The inverse wave generation algorithm for the frequency domain signal in the low frequency band is represented. The adjustable parameter information at least includes a weight coefficient corresponding to the ambient sound type to which the ambient audio signal belongs.

[0178] Exemplarily, the ambient sound type may be related to the frequency size of the ambient audio, and the ambient sound type may be divided into a high frequency band type, a mid-frequency band type, and a low frequency band type. Among them, the above-mentioned ambient sound type includes any one of a high frequency band type, a mid-frequency band type, and a low frequency band type. If the ambient sound type of the ambient audio signal is a high frequency band type, the high frequency band weight coefficient is determined as the adjustable parameter information, and the high frequency band weight coefficient is increased, so that the degree of concentration on the high frequency band audio in the ambient audio can be increased, thereby improving the reverse wave generation accuracy of the high frequency band audio in the ambient audio. If the ambient sound type of the ambient audio signal is a mid-frequency band type, the mid-frequency band weight coefficient is determined as the adjustable parameter information, and the mid-frequency band weight coefficient is increased, so that the degree of concentration on the mid-frequency band audio in the ambient audio can be increased, thereby improving the reverse wave generation accuracy of the mid-frequency band audio in the ambient audio. If the ambient sound type of the ambient audio signal is a low-frequency type, the low-frequency weight coefficient is determined as the adjustable parameter information, and the low-frequency weight coefficient is increased, so that the focus on the low-frequency audio in the ambient audio can be increased, thereby improving the reverse wave generation accuracy of the low-frequency audio in the ambient audio. Then, the digital signal processor uses a preset reverse wave generation algorithm to generate a reverse sound wave signal of the ambient audio signal based on the adjusted weight coefficient and the ambient audio signal.

[0179] Optionally, considering that for different audio streams, the playback duration of each frame of audio stream may be different, the playback duration is related to the sampling rate corresponding to the audio stream type selected by the application framework layer. Among them, when the playback duration is greater than the preset threshold, each frame of the audio stream can be split into multiple sub-audio streams, that is, the media audio signal can be split into multiple media sub-audio signals. This can shorten the time interval for sending the audio stream to the headset, thereby shortening the time interval for sending the target superimposed audio signal to the headset, and then reducing the delay in transmitting the target environmental audio signal to the headset, so as to ensure the correspondence between the sound wave signal corresponding to the target environmental audio signal in the target superimposed audio signal and the forward sound wave signal of the ambient audio directly entering the user's ear.

[0180] Exemplarily, in the process of generating the target superimposed audio signal, if the playback duration of the song audio stream transmitted each time is greater than a preset threshold, the song audio stream is first split into multiple song sub-audio streams. Then, for each song sub-audio stream, the target environmental audio signal and the song sub-audio stream are superimposed to obtain a superimposed sub-audio signal; and the superimposed sub-audio signal is sent to the earphone. Among them, the number of sampling points contained in the environmental audio signal is proportional to the playback duration of the song sub-audio stream. In other words, the above-mentioned target superimposed audio signal is split into multiple superimposed sub-audio signals, and the multiple superimposed sub-audio signals are sent to the earphones one after another, so that the transmission time interval of the superimposed sub-audio signal is relatively small, thereby shortening the time interval for sending the audio stream to the earphone.

[0181] Considering that both audio processing and audio transmission are time-consuming, this will cause the reverse sound wave signal or forward sound wave signal corresponding to the target ambient audio signal transmitted from the mobile phone to the headset to be delayed by the forward sound wave signal of the ambient audio directly entering the user's ear. Therefore, in order to further improve the correspondence between the two, thereby enhancing the user's listening experience of the noise reduction effect or the transparent effect, a part of the ambient audio signal can be cut out, and the cut audio signal will not be superimposed. Among them, the cut ambient audio signal can be understood as the ambient sound that has entered the user's ear during the time period of audio processing and audio transmission. In this way, only the target ambient audio signal corresponding to the cut ambient audio signal is superimposed with the song sub-audio stream to obtain a superimposed sub-audio signal, which can further ensure the correspondence between the reverse sound wave signal or forward sound wave signal corresponding to the target ambient audio signal and the sound wave signal of the ambient audio currently entering the user's ear.

[0182] Exemplarily, in the process of generating the superimposed sub-audio signal, the ambient audio signal is firstly clipped based on the ambient sound delay time to obtain a clipped ambient audio signal; wherein the ambient sound delay time is determined based on at least one of the generation time and the transmission time of the superimposed sub-audio signal. Then, the target ambient audio signal corresponding to the clipped ambient audio signal is superimposed on the song sub-audio stream to obtain a superimposed sub-audio signal.

[0183] It can be understood that, in the process of generating the target superimposed audio signal, the first implementation method only adds the process of splitting the song audio stream into multiple song sub-audio streams; the second implementation method only adds the process of cropping the ambient audio signal based on the ambient sound delay duration; the third implementation method adds the two processing processes of splitting the song audio stream into multiple song sub-audio streams and cropping the ambient audio signal based on the ambient sound delay duration. For the third implementation method, on the one hand, the signal delay caused by the two dimensions of audio signal transmission time interval and audio signal processing is taken into consideration to ensure the correspondence between the sound wave signal corresponding to the target ambient audio signal in the target superimposed audio signal and the forward sound wave signal of the ambient audio directly entering the user's ear; on the other hand, considering the cropping of the ambient audio signal, a part of the superimposed sub-audio signal will only contain media audio, but not the superimposed target ambient audio signal, which will cause the user to not feel the noise reduction or transparency effect; however, since the superimposed audio signal generated based on the song audio stream is split into multiple superimposed sub-audio signals, and the multiple superimposed sub-audio signals are sent to the headphones one after another, for each superimposed sub-audio signal, the number of cropped sampling points in the ambient audio signal is reduced, thereby reducing the user's listening experience of not feeling the noise reduction or transparency effect.

[0184] S611: After generating the target superimposed audio signal, the signal processing module sends the target superimposed audio signal to the Bluetooth module.

[0185] Exemplarily, when the ambient sound control mode is the noise reduction mode, the digital signal processor sends a first superimposed audio signal to the Bluetooth module; the first superimposed audio signal includes a reverse wave audio signal of the ambient audio and a sound wave signal of the media audio. When the ambient sound control mode is the transparent mode, the digital signal processor sends a second superimposed audio signal to the Bluetooth module; the second superimposed audio signal includes a forward wave audio signal of the ambient audio and a media audio signal.

[0186] S612, the Bluetooth module sends the target superimposed audio signal to the Bluetooth headset, wherein the Bluetooth headset plays the sound wave signal corresponding to the target superimposed audio signal through the headset speaker.

[0187] It should be noted that the earphones can be wireless earphones or wired earphones. When the earphones are wired earphones, the audio transmission interface is the headphone interface. The application processor in the mobile phone transmits the target superimposed audio signal to the earphones through the headphone interface. The specific implementation process of generating the target superimposed audio signal can be referred to the above process and will not be repeated here.

[0188] In some embodiments provided by the present application, the application layer sends the song audio stream and the ambient sound control mode identifier of the song being played by the music application to the application framework layer. The application framework layer sends the access address of the song audio stream and the ambient sound control mode identifier to the hardware abstraction layer. The hardware abstraction layer controls the microphone to power on and collect the ambient audio signal in real time through the kernel layer, and controls the digital signal processor to establish an audio playback link and an ambient sound feedback link, and sends the access address of the song audio stream and the ambient sound control mode identifier to the digital signal processor. The digital signal processor obtains the song audio stream from the application framework layer through the audio playback link, and obtains the ambient audio signal from the microphone through the ambient sound feedback link, and then generates a target superimposed audio signal based on the ambient sound control mode identifier, the song audio stream and the ambient audio signal, and transmits the target superimposed audio signal to the headphones.

[0189] In this way, since the generation process of the target superimposed audio signal is executed by the digital signal processor, the digital signal processor creates the audio playback link and the ambient sound feedback link under the triggering of the hardware abstraction layer; and if the ambient audio acquisition port specified in the ambient sound feedback link is the microphone port, the acquisition process of the ambient audio signal required for the generation of the target superimposed audio signal is directly obtained by the digital signal processor from the microphone. Therefore, the data flow of the ambient audio signal only needs to be from the microphone to the digital signal processor, without passing through the hardware abstraction layer and the application layer program framework layer in the software architecture involved in the application processor, which can greatly shorten the acquisition time of the ambient audio signal, thereby improving the correspondence between the reverse sound wave signal or the forward sound wave signal of the ambient audio played by the earphone and the sound wave signal of the ambient audio directly entering the user's ear, further enhancing the user's audio listening experience.

[0190] Scene 2

[0191] In this scenario, the ambient sound control function of the electronic device can not only provide the selection function of different ambient sound control modes (such as noise reduction mode or transparency mode), but also provide the function of adjusting the strength of the ambient sound control effect in a certain mode. Among them, during the song playback process, the user triggers the adjustment of the strength of the ambient sound control effect. At this time, the application layer needs to send down the strength adjustment amplitude of the ambient sound control effect, and the digital signal processor needs to increase or decrease the amplitude of the target ambient audio signal based on the strength adjustment amplitude, so that the noise reduction effect or transparency effect in the user's hearing sense changes. Exemplarily, taking the adjustment of the strength of the noise reduction effect as an example, the strength adjustment amplitude of the ambient sound control effect is the noise reduction amplitude.

[0192] Fig.10 FIG. 1 is a schematic diagram of an electronic device framework for performing an audio processing method. Fig. 9Based on the above, the setting application may include a mode setting module and an effect strength adjustment module. Fig.10 As shown in the figure, the main implementation process includes:

[0193] S701, when the music application is in the music playing state, it sends a song audio stream to the audio stream control service.

[0194] For example, during the music playback process, the song audio stream to be played is obtained, and the song audio stream is sent to the application framework layer at a preset time interval. Each transmitted song audio stream can be called an audio frame, and each audio frame contains a certain number of sampling points; wherein the preset time interval is related to the playback duration of each audio frame, and the number of sampling points is related to the audio sampling rate set when the audio stream is created.

[0195] S702: The mode setting module sends an ambient sound control mode identifier to the configuration detection service.

[0196] Wherein, based on the setting information of the ambient sound control mode, the ambient sound control mode identifier is determined. Exemplarily, the ambient sound control mode identifier may include an identifier of the noise reduction mode or an identifier of the transparency mode. If the ambient sound control mode identifier includes an identifier of the noise reduction mode, it means that the switch control of the noise reduction mode is in the on state and the switch control of the transparency mode is in the off state, and the ambient sound control mode is the noise reduction mode. If the ambient sound control mode identifier includes an identifier of the transparency mode, it means that the switch control of the transparency mode is in the on state and the switch control of the noise reduction mode is in the off state, and the ambient sound control mode is the transparency mode.

[0197] It is understandable that, since the ambient sound control mode identifier has been sent to the configuration detection service when the song starts playing, step S702 can be omitted during the song playing process.

[0198] S703, the effect strength adjustment module sends the noise reduction amplitude to the configuration detection service.

[0199] After detecting the user's triggering operation on the strength adjustment control of the ambient sound control effect, in response to the triggering operation, the strength adjustment amplitude of the ambient sound control effect in the currently enabled ambient sound control mode is determined. The strength adjustment amplitude of the ambient sound control effect can be based on the above Figures 6 to 8 The strength adjustment control may include a first movable member 506 on the noise reduction strength adjustment bar 505 .

[0200] For example, if the currently set ambient sound control mode is the noise reduction mode, the above-mentioned strength adjustment range is the noise reduction range. At this time, the larger the noise reduction range, the stronger the noise reduction effect; if the currently set ambient sound control mode is the transparency mode, the above-mentioned strength adjustment range is the transparency range. At this time, the larger the transparency range, the stronger the transparency effect.

[0201] It is understandable that when the song starts playing, the ambient sound control mode identifier has been sent to the application framework layer. When the ambient sound control mode has not been switched, that is, the ambient sound control mode identifier has not changed, only the noise reduction amplitude can be sent to the application framework layer, that is, the above step S702 can be not executed.

[0202] In addition, if the user has set the noise reduction amplitude before the song plays, then the noise reduction amplitude can also be sent to the application framework layer when the song starts playing; if the user adjusts the strength of the noise reduction effect during the song playback, then the application layer needs to send the latest noise reduction amplitude to the application framework layer.

[0203] That is to say, from the beginning to the end of the song playing, it is not necessary to send the noise reduction amplitude every time the song audio signal is sent to the application framework layer. Only when the noise reduction amplitude changes, the latest noise reduction amplitude will be sent to the application framework layer the next time the song audio signal is sent to the application layer.

[0204] S704: The configuration detection service sends the ambient sound control mode identifier and the noise reduction range to the mode control service.

[0205] For example, after the mobile phone establishes a Bluetooth connection with the headset, the application framework layer obtains the device attribute information of the headset through the hardware abstraction layer and the kernel layer, and determines whether the headset has the ambient sound control function based on the device attribute information. If the headset does not have the ambient sound control function, the relevant processing flow of executing the ambient sound control on the mobile phone side is triggered.

[0206] It is understandable that if the earphone connected to the mobile phone has not changed, there is no need to repeat the step of determining whether the earphone has the ambient sound control function. Next, if it is detected that a new earphone is connected to the mobile phone, it is necessary to again determine whether the earphone does not have the ambient sound control function based on the device attribute information of the newly connected earphone through the configuration detection service in the application framework layer, and then decide whether to trigger the relevant processing flow of executing the ambient sound control on the mobile phone side when sending the next frame of the song audio signal.

[0207] S705: The mode control service sends the ambient sound control mode identifier and the noise reduction amplitude to the audio HAL.

[0208] Exemplarily, the mode control service is responsible for sending the information transmitted by the configuration detection service to the hardware abstraction layer, and then transmitting it to the digital signal processor. The digital signal processor determines how to generate the target superimposed audio signal.

[0209] S706: The audio stream control service sends the access address of the song audio signal to the audio HAL.

[0210] For example, after receiving the song audio stream sent by the application layer, the audio stream control service first stores the song audio stream to a specified storage location and generates an access address for the corresponding song audio stream. In this way, it is only necessary to send the access address of the song audio stream to the hardware abstraction layer and the digital signal processor in sequence, and the digital signal processor can directly obtain the song audio stream based on the access address.

[0211] S707, the audio HAL sends a media audio acquisition instruction to the audio playback link to trigger the audio playback link to acquire the song audio stream from the application framework layer.

[0212] Exemplarily, the media audio acquisition instruction may carry the access address of the song audio stream. Since the audio signal processing process is performed by the digital signal processor, the application framework layer does not directly transmit the song audio stream to the hardware abstraction layer, but transmits the access address of the song audio stream, and then the hardware abstraction layer transmits the access address to the digital signal processor, so that the digital signal processor can directly obtain the corresponding song audio stream based on the access address request.

[0213] S708, the audio HAL sends an ambient audio acquisition instruction to the ambient sound feedback link to trigger the ambient sound feedback link to acquire the ambient audio signal from the microphone of the mobile phone.

[0214] The ambient audio signal is an audio signal obtained by converting the sound wave signal of the ambient audio collected in real time by the microphone. Since the audio signal processing process is performed by the digital signal processor, and when the song starts playing, the hardware abstraction layer has triggered the digital signal processor to create an ambient sound feedback link, the process of obtaining the ambient audio signal does not need to go through the hardware abstraction layer and the application layer program framework layer, but directly obtains the ambient audio signal from the microphone of the mobile phone through the ambient sound feedback link.

[0215] S709: The audio HAL sends an ambient sound control mode identifier and a noise reduction amplitude to the signal processing module.

[0216] It is understandable that the audio HAL drives the Bluetooth module to power on through the kernel layer when the mobile phone is connected to the headset, and the process of driving the microphone to power on can be performed when the song starts playing. During the song playing process, the Bluetooth module and the microphone are always in the on state, and there is no need to repeatedly drive the Bluetooth module and the microphone to power on. In addition, the audio playback link and the ambient sound feedback link can be created when the song starts playing. During the song playing process, the song audio stream can be directly obtained through the audio playback link, and the ambient audio signal can be obtained through the ambient sound feedback link, without repeatedly establishing the audio playback link and the ambient sound feedback link. In some example embodiments, in order to release memory usage, the audio playback link and the ambient sound feedback link can be destroyed after the song is played. If you need to play audio again, recreate the audio playback link and the ambient sound feedback link.

[0217] S710, the audio playback link transmits the song audio stream to the signal processing module.

[0218] S711, the ambient sound feedback link transmits the ambient audio signal to the signal processing module.

[0219] After receiving the ambient sound control mode identifier, the noise reduction amplitude, the song audio stream and the ambient sound signal, the signal processing module first generates an initial ambient audio signal according to the ambient sound control mode identifier and the ambient audio signal; wherein, when the ambient sound control mode is the noise reduction mode, the initial ambient audio signal includes an initial reverse wave audio signal corresponding to the ambient audio signal. Then, based on the noise reduction amplitude, the amplitude of the initial reverse wave audio signal is increased or decreased to obtain a reverse wave audio signal; and the reverse wave audio signal is determined as the target ambient audio signal.

[0220] In addition, if the ambient sound control mode is the transparent mode, the application layer sends the transparent amplitude to the application framework layer, so the signal processing module receives the transparent amplitude sent by the hardware abstraction layer. The initial ambient audio signal includes an initial positive wave audio signal corresponding to the ambient audio signal. Then, based on the transparent amplitude, the amplitude of the initial positive wave audio signal is increased or decreased to obtain a positive wave audio signal; the positive wave audio signal is determined as the target ambient audio signal.

[0221] Next, the target environment audio signal is superimposed on the song audio stream to obtain a target superimposed audio signal.

[0222] S712: After generating the target superimposed audio signal, the signal processing module sends the target superimposed audio signal to the Bluetooth module.

[0223] Exemplarily, when the ambient sound control mode is the noise reduction mode, the signal processing module sends a first superimposed audio signal to the Bluetooth module; the first superimposed audio signal includes a reverse wave audio signal of the ambient audio and a sound wave signal of the media audio. When the ambient sound control mode is the transparent mode, the signal processing module sends a second superimposed audio signal to the Bluetooth module; the second superimposed audio signal includes a forward wave audio signal of the ambient audio and a media audio signal.

[0224] S713, the Bluetooth module sends the target superimposed audio signal to the Bluetooth headset.

[0225] In some embodiments provided by the present application, when the headphones do not have an ambient sound control function, if the ambient sound control mode identifier includes an identifier of a noise reduction mode or an identifier of a transparency mode, the microphone is triggered to be turned on through the hardware abstraction layer and the kernel layer to collect ambient audio signals through the microphone, and the digital signal processor is triggered to create an audio playback link and an ambient sound feedback link, so that the digital signal processor obtains the song audio stream and the ambient audio signal from the microphone, and then generates a target superimposed audio signal based on the song audio stream and the ambient audio signal; and, by providing the user with a strength adjustment control for the ambient sound control effect, the user can use the adjustment control to personalize the strength of the noise reduction or transparency effect in combination with the actual audio listening effect and the current environment, so that the user can not only choose to turn on the noise reduction mode or the transparency mode, but also can adjust the strength of the noise reduction or transparency effect at any time.

[0226] It should be noted that for the specific implementation process in scenario 2, please refer to the explanation shown in scenario 1 above, which will not be repeated here.

[0227] Scene 3

[0228] In this scenario, although the ambient sound control mode of the electronic device is set to noise reduction mode or transparency mode, if it is detected that the headset has an ambient sound control function, the relevant processing flow of executing the ambient sound control on the mobile phone side will not be triggered.

[0229] Fig.11 The electronic device framework schematic diagram is exemplarily shown to perform the audio processing method. Fig.10 On the basis of Fig.11 As shown in the figure, the main implementation process includes:

[0230] S801, after detecting the user's triggering operation on the song playback control, the music application sends the song audio stream to the audio stream control service.

[0231] Exemplarily, after detecting a user's click operation on a playback control of a song, in response to the click operation, the song audio stream is obtained and sent to an audio stream control service.

[0232] S802: The configuration application sends an ambient sound control mode identifier to the configuration detection service.

[0233] The ambient sound control mode identifier is determined based on the setting information of the ambient sound control mode.

[0234] S803: When the configuration detection service detects that the headset has an ambient sound control function, the configuration detection service sends a single link creation indication message to the mode control service.

[0235] For example, before a song is played or when it starts playing, it is determined whether the headphones have the ambient sound control function based on the device attribute information of the headphones. For another example, during the playback of a song, if it is detected that a new headphone is connected, it is determined whether the headphones have the ambient sound control function based on the device attribute information of the newly connected headphone.

[0236] Exemplarily, when the headset does not have the ambient sound control function, the ambient sound control mode identifier and the strength adjustment amplitude of the ambient sound control effect are sent to the mode control service to trigger the relevant processing flow of executing the ambient sound control on the mobile phone side; and when the headset has the ambient sound control function, a single link creation indication message is sent to the mode control service, and the relevant processing flow of executing the ambient sound control on the mobile phone side will not be triggered.

[0237] S804: The mode control service sends the single link creation indication information to the audio HAL.

[0238] Among them, when the headset has an ambient sound control function, although the ambient sound control mode identifier sent by the application layer includes the noise reduction mode identifier or the transparency mode identifier, it is necessary to control the mobile phone to not trigger the ambient sound control function.

[0239] S805, the audio stream control service sends the access address of the song audio stream to the audio HAL.

[0240] S806: After receiving the single link creation indication information, the audio HAL sends a play link creation instruction to the digital signal processor to trigger the digital signal processor to create an audio play link.

[0241] Exemplarily, when a single-link creation indication message is received, only an instruction to create an audio playback link will be sent to the digital signal processor, and no microphone drive instruction will be sent to the audio driver in the kernel layer; no instruction to create an ambient sound feedback link will be sent to the digital signal processor, and there is no need to send an ambient sound control mode identifier to the signal processing module in the digital signal processor.

[0242] In the specific implementation, the process of determining whether the headset has the ambient sound control function is added to the application framework layer. If the headset does not have the ambient sound control function, and the ambient sound control mode identifier includes the identifier of the noise reduction mode or the identifier of the transparent mode, it is determined that the audio control link to be created includes the audio playback link and the ambient sound feedback link; correspondingly, in the case where the headset has the ambient sound control function, even if the ambient sound control mode identifier includes the audio playback link and the ambient sound feedback link, then the audio control link to be created only includes the audio playback link. In other words, only if the headset does not have the ambient sound control function, and the ambient sound control mode identifier includes the identifier of the noise reduction mode or the identifier of the transparent mode, will the audio playback link and the ambient sound feedback link be controlled to be created at the same time. In the case where the headset has the ambient sound control function, only the audio playback link will be created, and the ambient sound control function will not be triggered on the mobile phone at this time, so as to avoid the simultaneous ambient sound control on the mobile phone and the headset, which affects the user's audio listening experience.

[0243] It can be understood that the process of the audio HAL driving the Bluetooth module to power on through the kernel layer can be performed when the mobile phone establishes a connection with the headset. During the song playback process, the Bluetooth module is always in the on state, and there is no need to repeatedly drive the Bluetooth module to power on; in addition, when the song starts playing, the audio HAL sends a playback link creation instruction to the digital signal processor to control the digital signal processor to create the required audio playback link; then, during the song playback process, the audio playback link will not be destroyed, and there is no need to trigger the creation of the audio playback link again.

[0244] S807, the audio HAL sends the access address of the song audio stream to the audio playback link to trigger the audio playback link to obtain the song audio stream from the application framework layer.

[0245] S808, the audio playback link transmits the song audio stream to the signal processing module.

[0246] S809, the signal processing module sends the audio stream to be played to the Bluetooth module based on the above song audio stream.

[0247] Exemplarily, the signal processing module may not perform any processing on the song audio stream and directly send the song audio stream to the Bluetooth module, or it may perform preset processing on the song audio stream and then send the preset processed song audio stream to the Bluetooth module. The preset processing may include sound quality optimization processing and the like.

[0248] S810: The Bluetooth module sends the audio stream to be played to the Bluetooth headset.

[0249] In some embodiments provided by the present application, the application framework layer determines whether the headset has the ambient sound control function. If the headset has the ambient sound control function, the microphone will not be turned on. The application framework layer sends a single link creation indication message to the hardware abstraction layer. The hardware abstraction layer only triggers the digital signal processor to create an audio playback link. In this way, the ambient sound control function will not be triggered on the mobile phone end, so as to avoid the ambient sound control being performed on both the mobile phone end and the headset end at the same time, thereby affecting the user's audio listening experience.

[0250] In some possible implementations, the switching process of the ambient sound control mode can be manually implemented by the user, or the electronic device can automatically identify and switch the mode. The following describes in detail the implementation methods of these two mode switches in combination with Scene 4 and Scene 5.

[0251] Scene 4

[0252] In this scenario, during music playback, the user manually switches the ambient sound control mode, for example, the ambient sound control mode is switched from the noise reduction mode to the transparency mode. Fig.12 As shown, during the song playback, Fig.12 The song shown in (a) to (b) is played from 1 minute 30 seconds to 2 minutes 30 seconds; wherein, at 2 minutes 30 seconds, the user starts to trigger the mode switch (i.e., switching from the noise reduction mode to the transparency mode). For example, the user sets the ambient sound control mode through the control center interface, see Fig.12 In (b), a song playing interface is displayed. The user pulls down from the top right of the phone. The phone responds to the user's operation and displays the following Fig.12 The display interface of the control center shown in (c) in FIG. 1 shows a second switch control 507 (i.e., a control for setting the switch state of the transparency mode, which can be called the switch control of the transparency mode), a transparency strength adjustment bar 508, and a second movable member 509; wherein, the switch control of the noise reduction mode is in the on state, and the second switch control 507 is in the off state, and the ambient sound control mode is the noise reduction mode. The user clicks the second switch control 507; the mobile phone responds to the user operation, see Fig.12In (d), the switch control of the noise reduction mode is in the off state, and the second switch control 507 is in the on state, and the ambient sound control mode is the transparent mode. In addition, in the process of adjusting the strength of the transparent effect using the strength adjustment control, the user can adjust the strength of the transparent effect by moving the position of the second movable member 509 on the transparent strength adjustment bar 508.

[0253] It is understandable that after the switch control of the noise reduction mode changes from the on state to the off state, the position of the movable part on the strength adjustment bar of the noise reduction mode can remain unchanged, so that the next time the noise reduction mode is switched to, the strength of the noise reduction effect defaults to the value adjusted last time. The position of the movable part on the strength adjustment bar of the noise reduction mode can also be moved to the lowest end, so that the next time the noise reduction mode is switched to, the user needs to readjust the strength of the noise reduction effect. This application does not limit which specific method is used.

[0254] Next, the song audio stream continues to be transmitted from the application layer to the digital signal processor in sequence, and the ambient audio signal is transmitted from the microphone to the digital signal processor, and the ambient sound control mode identifier (such as the transparent mode identifier) ​​and the strength adjustment amplitude are transmitted from the application layer to the digital signal processor in sequence. Then, based on the ambient sound control mode identifier and the strength adjustment amplitude, the song audio stream and the ambient audio signal are processed to obtain the target superimposed audio signal. The specific implementation process can be found in the above Fig.10 , I will not go into details here.

[0255] Scene 5

[0256] In this scenario, during music playback, the mobile phone can automatically identify and switch the ambient sound control mode. For example, if it is recognized that the user needs to listen to the ambient sound, the ambient sound control mode can be switched from the noise reduction mode to the transparency mode; for example, if it is recognized that the user needs to block the ambient sound, the ambient sound control mode can also be switched from the transparency mode to the noise reduction mode.

[0257] Exemplarily, first obtain a first recognition result of an associated application corresponding to the scene type of the user's current environment, and obtain a second recognition result obtained by signal recognition of the ambient audio signal; based on at least one of the first recognition result and the second recognition result, determine the target association information corresponding to the scene type of the user's current environment. Different scene types correspond to different associated applications, and therefore, different scene types correspond to different first recognition results. Different scene types correspond to different ambient audio recognition dimensions, and therefore, different scene types correspond to different second recognition results. Then, determine whether a mode switch is required based on the first recognition result and the second recognition result; specifically, if it is determined based on the first recognition result and the second recognition result that the first mode switching condition is currently met, then determine that a mode switch is required; or, if it is determined based on the first recognition result and the second recognition result that the second mode switching condition is currently met, then determine that a mode switch is required.

[0258] In some example embodiments, when the current scene type belongs to the first category of scenes, if both the first recognition result and the second recognition result satisfy the ambient sound enhancement trigger condition, it is determined that the first mode switching condition is currently satisfied; if both the first recognition result and the second recognition result satisfy the ambient sound shielding trigger condition, it is determined that the second mode switching condition is currently satisfied. In other example embodiments, when the current scene type belongs to the second category of scenes, if either the first recognition result or the second recognition result satisfies the ambient sound enhancement trigger condition, it is determined that the first mode switching condition is currently satisfied; if either the first recognition result or the second recognition result satisfies the ambient sound shielding trigger condition, it is determined that the second mode switching condition is currently satisfied.

[0259] For the scenario of taking public transportation, Fig.13 As shown, see Fig.13 In (a), the switch control of the noise reduction mode is in the on state, and the switch control of the transparency mode is in the off state. At this time, the ambient sound control mode is determined to be the noise reduction mode. Taking the scenario of taking a bus as an example, which belongs to the first type of scenario, when the first recognition result and the second recognition result both indicate that the ambient sound enhancement trigger condition is met, it is determined that the first mode switching condition is currently met. Among them, in the case of noise reduction mode, the first recognition result and the second recognition result are obtained; based on the first recognition result and the second recognition result, it is determined whether the site broadcast time node has been reached; if both recognition results indicate that the site broadcast time node has been reached, it is determined that the first recognition result and the second recognition result both indicate that the ambient sound enhancement trigger condition is met, and at this time, the ambient sound control mode needs to be switched from the noise reduction mode to the transparency mode.

[0260] Exemplarily, if the station announcement prompt voice contains specified keywords (such as Dear Passenger, which is the preface of the station announcement), and the distance between the current location information of the user wearing the headset and the user's destination station meets the first constraint condition, it means that the user's destination station is approaching and the voice announcement starts. At this time, it can be determined that the station announcement time node has been reached and the ambient sound control mode needs to be automatically switched. Among them, the first recognition result may include the user's current location information from the map application and the user's travel destination station from the smart assistant application; the second recognition result may include the station announcement prompt voice obtained by performing ambient sound recognition based on the keyword recognition dimension. For example, if the distance between the user's current location information and the user's travel destination station is less than a preset threshold, it is determined that the first recognition result indicates that the ambient sound enhancement trigger condition is met; and if the station announcement prompt voice includes the preface of the station announcement, it is determined that the first recognition result indicates that the ambient sound enhancement trigger condition is met.

[0261] Specifically, if the first recognition result and the second recognition result both indicate that the current ambient sound enhancement trigger condition is met, then it is determined that the current first mode switching condition is met, that is, the current control mode needs to be switched from the noise reduction mode to the transparency mode, and the application framework layer sends the first mode switching information to the setting application, and the setting application switches the mode based on the first mode switching information. For example, see Fig.13 In (b), the switch control of the noise reduction mode is in the off state, and the switch control of the transparency mode is in the on state. At this time, the ambient sound control mode is the transparency mode.

[0262] It is understandable that the process of determining whether the mode switching condition is currently met based on the target association information can be performed by the application framework layer, specifically by the mode control service; it can also be performed by the digital signal processor, specifically by the signal processing module. Considering that the first recognition result comes from the associated application in the application layer, the second recognition result is obtained by the digital signal processor analyzing the ambient audio signal; and if it is detected that the ambient sound control mode needs to be switched, then it is necessary to send the mode switching information to the setting application. Exemplarily, the mode control service in the application framework layer determines whether the first mode switching condition is currently met based on the target association information. If the target association information includes the first recognition result and the second recognition result, it is only necessary for the application layer to send the first recognition result to the application framework layer, and the digital signal processor sends the second recognition result to the application framework layer through the hardware abstraction layer. The mode control service determines whether the first mode switching condition is currently met based on the first recognition result and the second recognition result; further, when determining whether the first mode switching condition is currently met, the application framework layer sends the first mode switching information to the setting application. In this way, the application framework layer does not need to send the first recognition result to the hardware abstraction layer and the digital signal processor, and the first mode switching information does not need to be sent by the digital signal processor to the setting application through the hardware abstraction layer and the application framework layer, which can reduce the transmission path of the first recognition result and the first mode switching information.

[0263] Next, after the current control mode is switched from the noise reduction mode to the transparency mode, the song audio stream continues to be transmitted from the application layer to the digital signal processor in sequence, and the ambient audio signal is transmitted from the microphone to the digital signal processor, and the ambient sound control mode identifier (such as the transparency mode identifier) ​​and the strength adjustment amplitude are transmitted from the application layer to the digital signal processor in sequence. Then, based on the ambient sound control mode identifier and the strength adjustment amplitude, the song audio stream and the ambient audio signal are processed to obtain the target superimposed audio signal. The specific implementation process can be found in the above Fig.10 , I will not go into details here.

[0264] Continue to see Fig.13In (b), the switch control of the noise reduction mode is in the off state, and the switch control of the transparent mode is in the on state. At this time, the ambient sound control mode is determined to be the transparent mode. Still taking the scenario of taking a bus as belonging to the first category of scenarios as an example, when the first recognition result and the second recognition result both indicate that the ambient sound shielding trigger condition is met, it is determined that the second mode switching condition is currently met. Among them, in the case of the transparent mode, the first recognition result and the second recognition result are obtained; based on the first recognition result and the second recognition result, it is determined whether the site broadcast end node is reached; if both recognition results indicate that the site broadcast end node is reached, it is determined that the first recognition result and the second recognition result both indicate that the ambient sound shielding trigger condition is met, and at this time, the ambient sound control mode needs to be switched from the transparent mode to the noise reduction mode.

[0265] Exemplarily, if the site announcement prompt voice contains specified keywords (such as "Please take your own belongings with you", which is the ending of the site announcement), and the distance between the current location information of the user wearing the headset and the user's destination site meets the second constraint condition, it means that the user's destination site has been reached and the site voice announcement has been completed. At this time, it can be determined that the site announcement end node has been reached, and the ambient sound control mode needs to be automatically switched. Among them, the first recognition result may include the user's current location information from the map application and the user's travel destination site from the smart assistant application; the second recognition result may include the site announcement prompt voice obtained by ambient sound recognition based on the keyword recognition dimension. For example, if the distance between the user's current location information and the user's travel destination site remains unchanged for a period of time greater than the preset period of time (indicating that the user is getting off the bus and exiting the station), it is determined that the first recognition result meets the ambient sound shielding trigger condition; and if the site announcement prompt voice includes the ending of the site announcement, it is determined that the second recognition result meets the ambient sound shielding trigger condition.

[0266] Specifically, if both the first recognition result and the second recognition result indicate that the current environmental shielding trigger condition is met, it is determined that the second mode switching condition is met, that is, the current control mode needs to be switched from the transparent mode to the noise reduction mode, and the application framework layer sends the second mode switching information to the setting application, and the setting application switches the mode based on the second mode switching information. For example, see Fig.13 In (c), the switch control of the noise reduction mode is in the on state, and the switch control of the transparency mode is in the off state. At this time, the ambient sound control mode is the noise reduction mode.

[0267] Optionally, the mode control service in the application framework layer determines whether the second mode switching condition is currently met based on the target association information. Since the target association information includes the first recognition result and the second recognition result, and the first recognition result comes from the associated application in the application layer, and the second recognition result is obtained by the digital signal processor analyzing the ambient audio signal, it is only necessary for the application layer to send the first recognition result to the application framework layer, and the digital signal processor sends the second recognition result to the application framework layer through the hardware abstraction layer. The mode control service determines whether the second mode switching condition is currently met based on the first recognition result and the second recognition result; further, when determining whether the second mode switching condition is currently met, the application framework layer sends the second mode switching information to the setting application. In this way, the application framework layer does not need to send the first recognition result to the hardware abstraction layer and the digital signal processor, and the second mode switching information does not need to be sent by the digital signal processor to the setting application through the hardware abstraction layer and the application framework layer, which can reduce the transmission path of the first recognition result and the second mode switching information.

[0268] Next, after the current control mode is switched from the transparent mode to the noise reduction mode, the song audio stream continues to be transmitted from the application layer to the digital signal processor in sequence, and the ambient audio signal is transmitted from the microphone to the digital signal processor, and the ambient sound control mode identifier (such as the noise reduction mode identifier) ​​and the strength adjustment amplitude are transmitted from the application layer to the digital signal processor in sequence. Then, based on the ambient sound control mode identifier and the strength adjustment amplitude, the song audio stream and the ambient audio signal are processed to obtain the target superimposed audio signal. The specific implementation process can be found in the above Fig.10 , I will not go into details here.

[0269] For office scenes, Fig.14 As shown, see Fig.14 In (a), the switch control of the noise reduction mode is in the on state, and the switch control of the transparency mode is in the off state. At this time, it is determined that the ambient sound control mode is the noise reduction mode. Taking the office scene in the office as an example, if any one of the first recognition result and the second recognition result indicates that the ambient sound enhancement trigger condition is met, it is determined that the first mode switching condition is currently met. Wherein, in the case of noise reduction mode, the first recognition result and the second recognition result are obtained; based on the first recognition result and the second recognition result, it is determined whether the user has the need to listen to the voice of colleagues; if any one of the recognition results indicates that the user has the need to listen to the voice of colleagues, it is determined that any one of the first recognition result and the second recognition result indicates that the ambient sound enhancement trigger condition is met. At this time, the ambient sound control mode needs to be switched from the noise reduction mode to the transparency mode.

[0270] For example, taking user A as an example, a colleague sends a message to user A through a chat application, informing that after finishing the work at hand in 15 minutes, the colleague will call user A to eat. Therefore, if the agreed time is reached, the user needs to hear the ambient sound clearly so that user A can hear the colleague calling his nickname at the first time. Among them, the first recognition result may include the current time information from the clock application and the agreed time information about the preset event in the message from the chat application. If it is determined based on the current time information and the agreed time information that the user is in a state of waiting for a colleague to call, it is determined that the ambient sound enhancement trigger condition is met.

[0271] For another example, a colleague goes to the workstation of user A who is wearing headphones to discuss the progress of a project. First, the colleague calls user A "Xiao Zhang". At this time, user A will respond. If a voice containing user A's voiceprint is recognized (for example, the user's reply "Well, manager, you say" is recognized); it means that the user has a need for dialogue, so the ambient sound control mode needs to be switched from noise reduction mode to transparency mode. Among them, the second recognition result may include an ambient audio voiceprint recognition result obtained by performing ambient sound recognition based on the voiceprint recognition dimension. If the ambient audio voiceprint recognition result indicates that a voice containing a preset user voiceprint is recognized, it is determined that there is a need for dialogue, that is, the ambient sound enhancement trigger condition is met.

[0272] Specifically, if any one of the first recognition result and the second recognition result indicates that the current ambient sound enhancement trigger condition is met, it is determined that the current first mode switching condition is met, that is, the current control mode needs to be switched from the noise reduction mode to the transparency mode, and the application framework layer sends the first mode switching information to the setting application, and the setting application switches the mode based on the first mode switching information. For example, see Fig.14 In (b), the switch control of the noise reduction mode is in the off state, and the switch control of the transparency mode is in the on state. At this time, the ambient sound control mode is the transparency mode.

[0273] It is understandable that the process of determining whether the mode switching condition is currently met based on the target association information can be performed by the application framework layer, specifically by the mode control service; it can also be performed by the digital signal processor, specifically by the signal processing module. Considering that the first recognition result comes from the associated application in the application layer, the second recognition result is obtained by the digital signal processor analyzing the ambient audio signal; and if it is detected that the ambient sound control mode needs to be switched, then it is necessary to send the mode switching information to the setting application. Exemplarily, the mode control service in the application framework layer determines whether the first mode switching condition is currently met based on the target association information. If the target association information includes the first recognition result and the second recognition result, it is only necessary for the application layer to send the first recognition result to the application framework layer, and the digital signal processor sends the second recognition result to the application framework layer through the hardware abstraction layer. The mode control service determines whether the first mode switching condition is currently met based on the first recognition result and the second recognition result; further, when determining whether the first mode switching condition is currently met, the application framework layer sends the first mode switching information to the setting application. In this way, the application framework layer does not need to send the first recognition result to the hardware abstraction layer and the digital signal processor, and the first mode switching information does not need to be sent by the digital signal processor to the setting application through the hardware abstraction layer and the application framework layer, which can reduce the transmission path of the first recognition result and the first mode switching information.

[0274] Next, after the current control mode is switched from the noise reduction mode to the transparency mode, the song audio stream continues to be transmitted from the application layer to the digital signal processor in sequence, and the ambient audio signal is transmitted from the microphone to the digital signal processor, and the ambient sound control mode identifier (such as the transparency mode identifier) ​​and the strength adjustment amplitude are transmitted from the application layer to the digital signal processor in sequence. Then, based on the ambient sound control mode identifier and the strength adjustment amplitude, the song audio stream and the ambient audio signal are processed to obtain the target superimposed audio signal. The specific implementation process can be found in the above Fig.10 , I will not go into details here.

[0275] Continue to see Fig.14In (b), the switch control of the noise reduction mode is in the off state, and the switch control of the transparency mode is in the on state. At this time, the ambient sound control mode is determined to be the transparency mode. Still taking the office scene in the office belonging to the second category as an example, when any one of the first recognition result and the second recognition result indicates that the ambient sound shielding trigger condition is met, it is determined that the second mode switching condition is currently met. Among them, in the case of the transparency mode, human voice recognition is performed on the ambient audio signal to obtain a human voice recognition result; if the time interval of the voice data containing the voiceprint of the user wearing headphones is greater than the preset threshold, it is determined that the conversation event is over; when the conversation event is over, it is determined that any one of the first recognition result and the second recognition result indicates that the ambient sound shielding trigger condition is met. At this time, the ambient sound control mode needs to be switched from the transparency mode to the noise reduction mode.

[0276] For example, if the user is in a lunch break and the current control mode is the transparent mode, the ambient sound control mode needs to be set to the noise reduction mode. The first recognition result may include current time information from the clock application. If it is determined based on the current time information that the user is in a lunch break, it is determined that the ambient sound shielding trigger condition is met.

[0277] For another example, after the user has finished talking with a colleague, the ambient sound control mode needs to be switched from the transparent mode to the noise reduction mode. The second recognition result may include an ambient audio voiceprint recognition result obtained by performing ambient sound recognition based on the voiceprint recognition dimension. If the ambient audio voiceprint recognition result indicates that the duration of the user's voiceprint speech not being detected is greater than the preset duration, it is determined that the conversation event is over, that is, the ambient sound enhancement shielding trigger condition is met.

[0278] If any one of the first recognition result and the second recognition result indicates that the current ambient sound shielding trigger condition is met, it is determined that the second mode switching condition is met, that is, the current control mode needs to be switched from the transparent mode to the noise reduction mode, and the application framework layer sends the second mode switching information to the setting application, and the setting application switches the mode based on the second mode switching information. For example, see Fig.14 In (c), the switch control of the noise reduction mode is in the on state, and the switch control of the transparency mode is in the off state. At this time, the ambient sound control mode is the noise reduction mode.

[0279] Optionally, the mode control service in the application framework layer determines whether the second mode switching condition is currently met based on the target association information. Since the target association information includes the first recognition result and the second recognition result, and the first recognition result comes from the associated application in the application layer, and the second recognition result is obtained by the digital signal processor analyzing the ambient audio signal, it is only necessary for the application layer to send the first recognition result to the application framework layer, and the digital signal processor sends the second recognition result to the application framework layer through the hardware abstraction layer. The mode control service determines whether the second mode switching condition is currently met based on the first recognition result and the second recognition result; further, when determining whether the second mode switching condition is currently met, the application framework layer sends the second mode switching information to the setting application. In this way, the application framework layer does not need to send the first recognition result to the hardware abstraction layer and the digital signal processor, and the second mode switching information does not need to be sent by the digital signal processor to the setting application through the hardware abstraction layer and the application framework layer, which can reduce the transmission path of the first recognition result and the second mode switching information.

[0280] Next, after the current control mode is switched from the transparent mode to the noise reduction mode, the song audio stream continues to be transmitted from the application layer to the digital signal processor in sequence, and the ambient audio signal is transmitted from the microphone to the digital signal processor, and the ambient sound control mode identifier (such as the noise reduction mode identifier) ​​and the strength adjustment amplitude are transmitted from the application layer to the digital signal processor in sequence. Then, based on the ambient sound control mode identifier and the strength adjustment amplitude, the song audio stream and the ambient audio signal are processed to obtain the target superimposed audio signal. The specific implementation process can be found in the above Fig.10 , I will not go into details here.

[0281] It should be noted that in other example embodiments, after automatically identifying the need for mode switching, a reminder message of mode switching is first sent to the user, and then based on the user's mode switching feedback information, it is determined whether to execute the mode switching. The specific implementation method can be set in combination with actual needs, and this application does not limit this. Exemplarily, after detecting the user's trigger operation on the music application, if the headset does not have a noise reduction function and the noise reduction function of the mobile phone is turned off, the user is prompted whether to set the noise reduction function to the on state; when it is determined that the noise reduction function is set to the on state, the media audio signal of the music application is obtained, and the ambient audio signal is collected through the built-in microphone of the mobile phone; based on the sound wave signal of the media audio signal and the reverse sound wave signal of the ambient audio signal, signal superposition processing is performed to obtain the target superimposed audio signal; then the target superimposed audio signal is transmitted to the headset, and the sound wave signal corresponding to the superimposed audio signal is played through the headset speaker.

[0282] Further, if the current ambient sound control mode is set to the transparent mode, the forward wave audio signal corresponding to the ambient audio signal can be directly generated. Exemplarily, considering that not all the audio in the ambient audio signal is what the user wants to hear; and considering that for different scene types, the audio that the user wants to hear is also different. Therefore, in combination with the scene type of the user's environment, the effective ambient audio in the ambient audio signal is distinguished, and it is determined which of the ambient audio signals belongs to the effective ambient audio (corresponding to the effective audio signal) and which belongs to the invalid ambient audio (corresponding to the invalid audio signal). Then, the forward wave audio signal of the effective ambient audio and the reverse wave audio signal of the invalid ambient audio in the ambient audio signal are generated. Therefore, the above-mentioned initial ambient audio signal includes the initial forward wave audio signal corresponding to the effective audio signal and the reverse wave audio signal corresponding to the invalid audio signal. The above-mentioned target ambient audio signal includes the forward wave audio signal corresponding to the effective audio signal and the reverse wave audio signal corresponding to the invalid audio signal. Then, the target ambient audio signal is superimposed with the song audio stream to obtain the target superimposed audio signal. This can improve the user's auditory experience in the transparent mode, so that the user can shield the ambient audio that the user does not need to hear, while enhancing the ambient audio that the user wants to hear.

[0283] It is understandable that in order to implement the audio processing method in the embodiment of the present application, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiment disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0284] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the audio processing method in the above-mentioned embodiment.

[0285] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the audio processing method in the above-mentioned embodiment.

[0286] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the audio processing method in the above-mentioned method embodiments.

[0287] In addition, an embodiment of the present application also provides a chip, which may include one or more processing circuits and one or more transceiver pins; wherein the transceiver pins and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the above-mentioned related method steps to implement the audio processing method in the above-mentioned embodiment to control the receiving pin to receive the signal, so as to control the sending pin to send the signal.

[0288] Among them, the electronic device (such as a mobile phone, etc.), computer storage medium, computer program product, device or chip, etc. provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0289] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0290] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0291] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An audio processing system, characterized in that: The system comprises an electronic device and a headset, wherein the electronic device comprises a microphone, wherein the control mode of the electronic device for ambient sound comprises a noise reduction mode and a transparency mode, and the headset does not have an ambient sound control function; the electronic device and the headset are connected by wire or wirelessly; The electronic device is used to obtain a media audio signal of a preset application; and to collect an ambient audio signal in real time through the microphone; The electronic device is further used to generate a first audio signal according to the current control mode and the ambient audio signal; perform signal superposition processing on the first audio signal and the media audio signal to obtain a second audio signal, and send the second audio signal to the headset; wherein, when the current control mode is the noise reduction mode, the first audio signal includes a reverse wave audio signal corresponding to the ambient audio signal, and when the current control mode is the transparent mode, the first audio signal includes a forward wave audio signal corresponding to the ambient audio signal; The earphone is used to receive and play the second audio signal; The electronic device is also used to identify whether the ambient sound mode is switched based on the target association information corresponding to the scene type of the current environment through the application framework layer, and trigger the setting application in the application layer to update the current control mode when the mode switch is determined; the target association information includes a first recognition result and a second recognition result; the first recognition result comes from the associated application corresponding to the scene type in the application layer, and the second recognition result is obtained by the digital signal processor of the electronic device analyzing the ambient audio signal.

2. The system according to claim 1, characterized in that The electronic device is specifically configured to generate a third audio signal according to a current control mode and the ambient audio signal; wherein, when the current control mode is a noise reduction mode, the third audio signal includes an initial reverse wave audio signal corresponding to the ambient audio signal, and when the current control mode is a transparent mode, the third audio signal includes an initial forward wave audio signal corresponding to the ambient audio signal; The electronic device is specifically used to increase or decrease the amplitude of the initial reverse wave audio signal according to the noise reduction amplitude of the noise reduction mode to obtain the reverse wave audio signal; or to increase or decrease the amplitude of the initial forward wave audio signal according to the transparency amplitude of the transparency mode to obtain the forward wave audio signal.

3. The system according to claim 2, characterized in that The electronic device is specifically used to perform an ambient sound type analysis according to the ambient audio signal to determine the ambient sound type if the current control mode is a noise reduction mode; and determine the adjustable parameter information of a preset reverse wave generation algorithm according to the ambient sound type; The initial reverse wave audio signal is generated according to the adjustable parameter information and the ambient audio signal by using the preset reverse wave generation algorithm.

4. The system according to claim 1, characterized in that The electronic device is further used to identify the scene type of the user's environment; and obtain target association information corresponding to the scene type; If it is determined according to the target association information that the first mode switching condition is currently met, the current control mode is switched from the noise reduction mode to the transparent mode; If it is determined according to the target association information that the second mode switching condition is currently met, the current control mode is switched from the transparent mode to the noise reduction mode.

5. The system according to claim 4, characterized in that The electronic device is also used to determine the valid audio signal and the invalid audio signal in the ambient audio signal according to the scene type if the current control mode is the transparent mode; wherein the first audio signal includes a forward wave audio signal corresponding to the valid audio signal and a reverse wave audio signal corresponding to the invalid audio signal.

6. The system according to claim 1, characterized in that The electronic device also includes an application processor and a digital signal processor; The application processor is used to obtain a media audio signal of a preset application and determine a mode identifier of the current control mode; The application processor is further configured to drive the microphone to collect ambient audio signals in real time; and to send the first indication information and the mode identifier to the digital signal processor; The digital signal processor is used to create an audio playback link and a first feedback link according to the first indication information; obtain the media audio signal from the application processor through the audio playback link; and obtain the ambient audio signal from the microphone through the first feedback link; The digital signal processor is further configured to generate a first audio signal according to the mode identifier and the ambient audio signal.

7. The system according to claim 6, characterized in that The application processor is further configured to send a microphone start instruction to the headset when detecting that the electronic device enters the pocket mode; and send second indication information to the digital signal processor; The earphone is further used to respond to the microphone start instruction, turn on the microphone of the earphone, so as to collect the ambient audio signal in real time through the microphone; and transmit the ambient audio signal to the audio transmission interface of the electronic device; The digital signal processor is further configured to destroy the first feedback link and create a second feedback link according to the second indication information, so as to obtain the ambient audio signal from the audio transmission interface through the second feedback link.

8. The system according to any one of claims 1 to 7, characterized in that: The electronic device is further used to determine whether the headset has an ambient sound control function according to the device attribute information of the headset after establishing a connection with the headset; wherein the device attribute information of the headset includes a headset device identifier; If the earphone device identifier does not belong to the preset earphone whitelist, it is determined that the earphone does not have the ambient sound control function.

9. An audio processing method, characterized in that: Applied to an electronic device, the electronic device includes a microphone, the control mode of the electronic device for ambient sound includes a noise reduction mode and a transparency mode, and a headset connected to the electronic device by wire or wirelessly does not have an ambient sound control function, the method includes: Acquire a media audio signal of a preset application; and collect an ambient audio signal in real time through the microphone; Generate a first audio signal according to the current control mode and the ambient audio signal; wherein, when the current control mode is a noise reduction mode, the first audio signal includes a reverse wave audio signal corresponding to the ambient audio signal, and when the current control mode is a transparent mode, the first audio signal includes a forward wave audio signal corresponding to the ambient audio signal; Performing signal superposition processing on the first audio signal and the media audio signal to obtain a second audio signal; sending the second audio signal to the headset; The method also includes: identifying whether the ambient sound mode is switched based on the target association information corresponding to the scene type of the current environment through the application framework layer, and triggering the setting application in the application layer to update the current control mode when the mode switch is determined; the target association information includes a first recognition result and a second recognition result; the first recognition result comes from the associated application corresponding to the scene type in the application layer, and the second recognition result is obtained by the digital signal processor of the electronic device analyzing the ambient audio signal.

10. The method according to claim 9, characterized in that The step of generating a first audio signal according to the current control mode and the ambient audio signal comprises: Generate a third audio signal according to the current control mode and the ambient audio signal; wherein, when the current control mode is the noise reduction mode, the third audio signal includes an initial reverse wave audio signal corresponding to the ambient audio signal, and when the current control mode is the transparent mode, the third audio signal includes an initial forward wave audio signal corresponding to the ambient audio signal; According to the noise reduction amplitude of the noise reduction mode, the amplitude of the initial reverse wave audio signal is increased or decreased to obtain the reverse wave audio signal; or, according to the transparency amplitude of the transparency mode, the amplitude of the initial forward wave audio signal is increased or decreased to obtain the forward wave audio signal.

11. The method according to claim 10, characterized in that The step of generating a third audio signal according to the current control mode and the ambient audio signal comprises: If the current control mode is the noise reduction mode, performing an ambient sound type analysis according to the ambient audio signal to determine the ambient sound type; Determining adjustable parameter information of a preset reverse wave generation algorithm according to the type of ambient sound; The initial reverse wave audio signal is generated according to the adjustable parameter information and the ambient audio signal by using the preset reverse wave generation algorithm.

12. The method according to claim 9, characterized in that The method further comprises: Identify the scene type of the user's environment; and obtain target association information corresponding to the scene type; If it is determined according to the target association information that the first mode switching condition is currently met, the current control mode is switched from the noise reduction mode to the transparent mode; If it is determined according to the target association information that the second mode switching condition is currently met, the current control mode is switched from the transparent mode to the noise reduction mode.

13. The method according to claim 12, characterized in that After identifying the scene type of the user's environment, it also includes: If the current control mode is the transparent mode, the valid audio signal and the invalid audio signal in the ambient audio signal are determined according to the scene type; wherein the first audio signal includes a forward wave audio signal corresponding to the valid audio signal and a reverse wave audio signal corresponding to the invalid audio signal.

14. The method according to claim 9, characterized in that The electronic device also includes an application processor and a digital signal processor; The step of obtaining a media audio signal of a preset application; And collecting the ambient audio signal in real time through the microphone, including: Acquire a media audio signal of a preset application through the application processor, and determine a mode identifier of the current control mode; drive the microphone through the application processor to collect environmental audio signals in real time; and send first indication information and the mode identifier to the digital signal processor; create an audio playback link and a first feedback link through the digital signal processor according to the first indication information; and acquire the media audio signal from the application processor through the audio playback link; and acquire the environmental audio signal from the microphone through the first feedback link; The step of generating a first audio signal according to the current control mode and the ambient audio signal comprises: A first audio signal is generated by the digital signal processor according to the mode identifier and the ambient audio signal.

15. The method according to claim 14, characterized in that The method further comprises: When it is detected that the electronic device enters the pocket mode, sending a microphone start instruction to the headset through the application processor; and sending second indication information to the digital signal processor; The digital signal processor destroys the first feedback link and creates a second feedback link according to the second indication information, so as to obtain an ambient audio signal from the audio transmission interface of the electronic device through the second feedback link; the ambient audio signal is an audio signal that the headset controls the microphone of the headset to collect in real time and transmit to the audio transmission interface in response to the microphone start instruction.

16. The method according to any one of claims 9 to 15, characterized in that Before collecting the ambient audio signal in real time through the microphone, the method further includes: After establishing a connection with the headset, judging whether the headset has an ambient sound control function according to device attribute information of the headset; wherein the device attribute information of the headset includes a headset device identifier; If the earphone device identifier does not belong to the preset earphone whitelist, it is determined that the earphone does not have the ambient sound control function.

17. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the method as claimed in any one of claims 9 to 16.

18. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program is executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 9 to 16.

19. A chip, applied to an electronic device, comprising one or more processors, characterized in that: The processor is used to call computer instructions so that the electronic device executes the method according to any one of claims 9 to 16.

Images