Audio signal processing method and electronic device

By generating masking signals in the mobile phone and performing sound field control processing, the problem of sound leakage during the call is solved, the protection of user privacy and experience is improved, and the security and clarity of user conversation content is ensured.

CN117714581BActive Publication Date: 2024-11-01HONOR DEVICE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311017599.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-08-11
Publication Date
2024-11-01
Estimated Expiration
2043-08-11

AI Technical Summary

Technical Problem

During the call, the phone's sound leakage will lead to the user's conversation content being leaked, affecting the user's privacy protection and experience.

Method used

By generating a masking signal and performing sound field control processing on the target audio signal and masking signal, it is ensured that the sound field energy formed by the signal to be played in the ear area is greater than the sound field energy formed by the sound leakage area, thereby reducing sound leakage, protecting user privacy and Improve user experience.

Benefits of technology

Effectively reduce sound leakage, protect user privacy, improve user experience, and ensure that the content of the user's conversation during the call is not easy to be heard by others, and at the same time, it does not affect the user's hearing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117714581B_ABST
    Figure CN117714581B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides a method for processing an audio signal and an electronic device, the method is applied to an electronic device, the electronic device includes N target audio playback devices, N is an integer greater than or equal to 2, the method includes: obtaining a target audio signal, the target audio signal is an audio signal that needs to be played through the N target audio playback devices; generating a masking signal according to the target audio signal; performing sound field control processing on the target audio signal and the masking signal respectively, and then superimposing processing to obtain N channels of to-be-played signals corresponding to the N target audio playback devices; playing the corresponding to-be-played signals respectively through the N target audio playback devices; the energy of the sound field formed by the to-be-played signal in the first spatial region is greater than the energy of the sound field formed by the to-be-played signal in the second spatial region, and the first spatial region is closer to the target audio playback device than the second spatial region. The method can protect user privacy and improve user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to an audio signal processing method and electronic equipment. Background Art

[0002] Talking via mobile phones and other electronic devices is a common application scenario in people's lives. However, during the call, the sound leakage of the mobile phone will cause the user's conversation content to be leaked, which is not conducive to the user's privacy protection. Summary of the invention

[0003] The present application provides an audio signal processing method and electronic device, which can reduce sound leakage, protect user privacy, and improve user experience.

[0004] In a first aspect, the present application provides a method for processing an audio signal, which is applied to an electronic device, wherein the electronic device includes N target audio playback devices, where N is an integer greater than or equal to 2, and the method includes: obtaining a target audio signal, where the target audio signal is an audio signal that needs to be played through the N target audio playback devices; generating a masking signal based on the target audio signal; performing sound field control processing on the target audio signal and the masking signal respectively, and then superimposing them to obtain N channels of signals to be played corresponding to the N target audio playback devices one by one; playing the corresponding signals to be played through the N target audio playback devices respectively; the energy of the sound field formed by the signal to be played in the first spatial area is greater than the energy of the sound field formed by the signal to be played in the second spatial area, wherein the first spatial area is closer to the N target audio playback devices than the second spatial area.

[0005] Optionally, the target audio playback device may be different due to the different electronic device structures and audio system settings. In a specific embodiment, the electronic device includes a receiver (i.e., a handset) and a speaker, the speaker is used to play audio in hands-free mode or in handheld mode, and the receiver is used to play audio in handheld mode. In this case, the target audio playback device is the receiver. In another specific embodiment, the electronic device only includes a speaker, and the speaker is used to play audio in both handheld mode and hands-free mode. In this case, the target audio playback device is a speaker.

[0006] Optionally, the target audio signal may be a call downlink signal output by a call application, a voice audio signal output by an instant messaging application, or an audio signal output by other applications, such as a recording audio signal output by a recording application. The present embodiment does not limit the specific source of the target audio signal.

[0007] Optionally, the first spatial area may be an in-ear area, and the second control area may be a sound leakage area.

[0008] The first aspect of the present application provides a method for processing an audio signal, generating a masking signal based on a target audio signal, and performing sound field control processing on the target audio signal and the masking signal respectively, and then superimposing the processed signals to obtain N channels of signals to be played; after the N channels of signals to be played are played through N target audio playback devices, the energy of the sound field formed in the first spatial area is greater than the energy of the sound field formed by the signal to be played in the second spatial area. Since the first spatial area is closer to the N target audio playback devices than the second spatial area, when the user uses an electronic device to listen to audio (for example, answering a call), the user is closer to the target audio playback device than the people around him. Therefore, the sound heard by the people around him (i.e., sound leakage) is smaller than the sound heard by the user, so it is possible to reduce sound leakage, protect user privacy, and improve user experience. In addition, the masking signal can psychologically mask the target audio signal and reduce the sound intelligibility, thereby further protecting user privacy and improving user experience.

[0009] In a possible implementation, the first spatial area is a spatial area within a preset range around N target audio playback devices, the second spatial area does not overlap with the first spatial area, and the second spatial area is located on a side of the first spatial area away from the electronic device.

[0010] The first spatial area may be the intra-ear area, i.e., when the target audio playback device of the electronic device is close to the human ear, the area around the electronic device including the human ear canal and a preset range around the ear. The second spatial area may be the sound leakage area, i.e., the area around the first spatial area, away from one side of the electronic device.

[0011] That is to say, the energy of the sound field formed by the signal to be played in the ear area (i.e., the intra-ear sound field) is greater than the energy of the sound field formed by the signal to be played in the sound leakage area (i.e., the sound leakage sound field). In this way, the sound heard by people around is smaller than the sound heard by the user, which reduces sound leakage, protects user privacy, and improves user experience.

[0012] In a possible implementation, a target audio signal and a masking signal are subjected to sound field control processing respectively and then superimposed to obtain N channels of signals to be played corresponding to the N target audio playback devices, including: performing a first sound field control processing on the target audio signal to obtain N channels of target output signals corresponding to the N target audio playback devices, wherein the energy of the sound field formed by the target output signal in the first spatial region is greater than the energy of the sound field formed in the second spatial region; performing a second sound field control processing on the masking signal to obtain N channels of masked output signals corresponding to the N target audio playback devices, wherein the energy of the sound field formed by the masked output signal in the second spatial region is greater than the energy of the sound field formed in the first spatial region; and superimposing the target output signal corresponding to each target audio playback device with the corresponding masking output signal to obtain N channels of signals to be played corresponding to the N target audio playback devices.

[0013] In this implementation, on the one hand, by performing sound field control on the target audio signal, the energy of the sound field finally formed by the target audio signal in the first spatial area is greater than the energy of the sound field formed in the second spatial area. In this way, sound leakage is reduced, user privacy is protected, and user experience is improved. On the other hand, by performing sound field control on the masking signal, the energy of the sound field finally formed by the masking signal in the second spatial area is greater than the energy of the sound field formed in the first spatial area. In this way, a smaller degree of psychoacoustic masking is performed on the target audio signal in the first spatial area, and a larger degree of psychological masking is performed on the target audio signal in the second spatial area, thereby reducing the intelligibility of the sound in the second spatial area while minimizing the impact on the first spatial area, that is, while protecting the user's privacy, reducing the impact on the user's hearing and improving the user experience.

[0014] In one possible implementation, the electronic device includes a first filter group and a second filter group; a first sound field control processing is performed on a target audio signal to obtain N target output signals corresponding one-to-one to N target audio playback devices, including: performing a first sound field control processing on the target audio signal through the first filter group to obtain N target output signals; a second sound field control processing is performed on a masking signal to obtain N masking output signals corresponding one-to-one to the N target audio playback devices, including: performing a second sound field control processing on the masking signal through the second filter group to obtain N masking output signals.

[0015] That is, the first filter group is a filter group for performing sound field control on the target audio signal, and the second filter group is a filter group for performing sound field control on the masking signal. The first filter group includes N first filters corresponding to the target audio playback devices one by one. The second filter group includes N second filters corresponding to the target audio playback devices one by one.

[0016] In one possible implementation, the first filter group is a filter group generated by taking the first spatial area as the bright area and the second spatial area as the dark area, performing a sound field control test or simulation, maximizing the sound field energy ratio of the bright area and the dark area, and solving the filter coefficient; the second filter group is a filter group generated by taking the second spatial area as the bright area and the first spatial area as the dark area, performing a sound field control test or simulation, maximizing the sound field energy ratio of the bright area and the dark area, and solving the filter coefficient.

[0017] That is to say, when the first filter group and the second filter group are solved and generated, the bright area and the dark area are swapped. The first filter group generated in this way can make the energy of the generated sound field (the in-ear sound field of the target audio signal) formed by the target audio signal in the first spatial area larger, and the energy of the generated sound field (the leakage sound field of the masking signal) formed by the target audio signal in the second spatial area smaller; the second filter group generated in this way can make the energy of the sound field (the in-ear sound field of the masking signal) generated by the masking signal in the first spatial area smaller, and the energy of the sound field (the leakage sound field of the masking signal) generated by the masking signal in the second spatial area larger. In this way, after the target audio signal and the masking signal are superimposed, the intelligibility of the in-ear sound field of the final output sound is not affected, while the intelligibility of the leakage sound field is reduced, so that the user's privacy is effectively protected without affecting the user's hearing sense, and the user experience is improved.

[0018] In a possible implementation, the frequency spectrum of the masking signal is consistent with that of the target audio signal, so that the masking signal has a better masking effect on the target audio signal and a better protection effect on user privacy.

[0019] In a possible implementation, generating a masking signal according to a target audio signal includes: generating an initial masking signal consistent with a spectrum of the target audio signal according to the target audio signal; and performing spectrum correction on the initial masking signal to obtain the masking signal.

[0020] In this implementation, the initial masking signal is spectrally corrected to generate a masking signal, and then the masking signal is subjected to sound field control, so that the spectrum distortion during the sound field control can be corrected, the masking effect is improved, and the final privacy protection effect is improved.

[0021] In a possible implementation, an initial masking signal consistent with the frequency spectrum of the target audio signal is generated according to the target audio signal, including: caching the target audio signal with a preset masking frame length to obtain the signal to be masked in the current frame; windowing the signal to be masked in the current frame and performing time reversal to obtain the inversion signal of the current frame; obtaining the inversion signal of the previous frame, which refers to the inversion signal of the previous frame that is closest to the inversion signal of the current frame; superimposing the inversion signal of the current frame with the inversion signal of the previous frame to obtain the basic masking signal of the current frame; adjusting the energy of the basic masking signal of the current frame according to a preset energy ratio threshold and the energy of the signal to be masked in the current frame to obtain the initial masking signal of the current frame.

[0022] Optionally, the preset masking frame length is greater than the frame length of the target audio signal. For example, the preset masking frame length may be a value between 50 ms and 300 ms.

[0023] In this implementation, the target audio signal is processed in real time to obtain an initial masking signal, so that the spectrum consistency of the final masking signal and the target audio signal is higher, the masking effect of the masking signal is improved, and thus the effect of privacy protection is improved. Moreover, the current frame inversion signal is superimposed with the previous frame inversion signal, which can improve the smoothness of the current basic masking signal obtained, thereby improving the smoothness of the final masking signal.

[0024] In a possible implementation, the energy of the basic masking signal of the current frame is adjusted according to a preset energy ratio threshold and the energy of the signal to be masked of the current frame to obtain the initial masking signal of the current frame, including: adjusting the energy of the basic masking signal of the current frame to obtain the masking adjustment signal of the current frame, and the energy ratio of the signal to be masked of the current frame to the masking adjustment signal of the current frame is equal to the preset energy ratio threshold; determining whether there is a valid signal in the signal to be masked of the current frame and the masking adjustment signal of the current frame respectively; if there is no valid signal in the signal to be masked of the current frame, reducing the amplitude of the masking adjustment signal of the current frame to obtain the initial masking signal of the current frame, or taking 0 or a null value as the initial masking signal of the current frame; if there is a valid signal in the signal to be masked of the current frame and there is no valid signal in the masking adjustment signal of the current frame, obtaining a preset signal from a preset masking signal library, adjusting the spectrum of the preset signal according to the spectrum of the signal to be masked of the current frame, and obtaining the initial masking signal of the current frame.

[0025] In this implementation, if there is no valid signal in the signal to be masked in the current frame, the amplitude of the mask adjustment signal of the current frame is reduced to obtain the initial mask signal of the current frame, or 0 or null value is used as the initial mask signal of the current frame, and the frame signal to be masked is not masked subsequently. In this way, excessive masking of the target audio signal of the signal without sound or with low sound is prevented, and the user's hearing sense is prevented from being affected. If there is a valid signal in the signal to be masked in the current frame, and there is no valid signal in the mask adjustment signal of the current frame, a preset signal is obtained from a preset mask signal library, and the spectrum of the preset signal is adjusted according to the spectrum of the signal to be masked in the current frame to obtain the initial mask signal of the current frame. If there is no valid signal in the mask adjustment signal of the current frame, it means that the signal is a silent signal or a signal with low sound, and the signal cannot effectively mask the frame signal to be masked. Therefore, the preset signal is directly obtained from the preset mask signal library, and the spectrum of the preset signal is adjusted to be consistent with the spectrum of the signal to be masked in the current frame, and the initial mask signal of the current frame is obtained. This can prevent the occurrence of invalid masking phenomenon and improve the masking effect.

[0026] In one possible implementation, it is determined whether there are valid signals in the current frame signal to be masked and the current frame mask adjustment signal, including: determining whether the energy of the current frame signal to be masked is greater than or equal to a first energy threshold; if the energy of the current frame signal to be masked is greater than or equal to the first energy threshold, determining that there is a valid signal in the current frame signal to be masked; if the energy of the current frame signal to be masked is less than the first energy threshold, determining that there is no valid signal in the current frame signal to be masked; determining whether the energy of the current frame mask adjustment signal is greater than or equal to a second energy threshold; if the energy of the current frame mask adjustment signal is greater than or equal to the second energy threshold, determining that there is a valid signal in the current frame mask adjustment signal; if the energy of the current frame mask adjustment signal is less than the second energy threshold, determining that there is no valid signal in the current frame mask adjustment signal.

[0027] In a possible implementation, generating a masking signal according to the target audio signal includes: acquiring a preset signal from a preset masking signal library; and adjusting a spectrum of the preset signal according to a spectrum of the target audio signal to obtain the masking signal.

[0028] In this implementation, a signal is acquired from a preset masking signal library to generate a masking signal, so that the masking signal can be obtained simply and quickly, and the algorithm is simplified.

[0029] In a second aspect, the present application provides a device, which is included in an electronic device, and the device has the function of implementing the electronic device behavior in the above-mentioned first aspect and the possible implementation of the above-mentioned first aspect. The function can be implemented by hardware, or it can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above-mentioned functions. For example, a receiving module or unit, a processing module or unit, etc.

[0030] In a third aspect, the present application provides an electronic device, comprising: a processor, a memory, and an interface; the processor, the memory, and the interface cooperate with each other so that the electronic device executes any one of the methods in the technical solution of the first aspect.

[0031] In a fourth aspect, the present application provides a chip, including a processor. The processor is used to read and execute a computer program stored in a memory to perform the method in the first aspect and any possible implementation thereof.

[0032] Optionally, the chip also includes a memory, and the memory is connected to the processor via a circuit or wire.

[0033] Further optionally, the chip also includes a communication interface.

[0034] In a fifth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes any one of the methods in the technical solution of the first aspect.

[0035] In a sixth aspect, the present application provides a computer program product, the computer program product comprising: a computer program code, when the computer program code runs on an electronic device, the electronic device executes any one of the methods in the technical solution of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application;

[0037] Figure 2 is an audio system architecture diagram of an electronic device 100 provided in an embodiment of the present application;

[0038] Figure 3 is a flowchart of an example of an audio signal processing method provided in an embodiment of the present application;

[0039] Figure 4 is a schematic diagram of the principle of an audio signal processing method provided in an embodiment of the present application;

[0040] Figure 5 is a flowchart of another audio signal processing method provided in an embodiment of the present application;

[0041] Figure 6 is a flowchart of another method for processing an audio signal provided in an embodiment of the present application;

[0042] Figure 7 is a schematic diagram of a sound field sampling point distribution provided in an embodiment of the present application;

[0043] Figure 8 This is a diagram of the corresponding relationship between the first filter group and the second filter group when solving and generating a sound field, provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0045] In the following, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features.

[0046] References to "one embodiment" or "some embodiments" etc. described in the specification of the present application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in the specification of the present application do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0047] In order to better understand the embodiments of the present application, the terms or concepts that may be involved in the embodiments are explained below.

[0048] The sound field refers to the space occupied by the elastic medium in which sound waves exist, that is, the part of the medium in which sound waves propagate.

[0049] Sound field control refers to the reproduction of the spatial sound field or the control of the spatial sound field distribution pattern by adjusting the amplitude and phase of the input signals of each audio playback device.

[0050] TMR is the energy ratio of the target signal to the masker signal (target-to-masker ratio, abbreviated as TMR).

[0051] The audio signal processing method provided in the embodiment of the present application can be applied to electronic devices with earpieces (receivers) and / or speakers, such as mobile phones, tablet computers, wearable devices, and personal digital assistants (PDAs). Alternatively, the audio signal processing method provided in the embodiment of the present application can also be applied to audio playback devices such as headphones. The embodiment of the present application does not impose any restrictions on the specific type of electronic device, and the following mainly takes mobile phones as an example for explanation.

[0052] For example, Figure 11 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0053] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0054] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), an audio processor (ADSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0055] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0056] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0057] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0058] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0059] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to implement the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0060] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0061] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0062] The wireless communication function of the electronic device can be implemented by antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna. For example, antenna 1 can be reused as a diversity antenna of a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0063] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to electronic devices. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0064] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio playback device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0065] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0066] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0067] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the ADSP.

[0068] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0069] Speaker 170A, also called "speaker", is used to convert audio electrical signals into sound signals. Electronic device 100 can play music, or play hands-free calls or voice messages through speaker 170A, that is, play audio in hands-free mode. Optionally, speaker 170A can also be used to play audio in handheld mode.

[0070] The receiver 170B, also called a "receiver", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message in the handheld mode, the user can listen to the voice by placing the receiver 170B close to the ear. That is, the receiver 170B can be used to play audio in the handheld mode.

[0071] It is understood that in some embodiments, the electronic device 100 may not include the receiver 170B, but only include the speaker 170A. In this case, the speaker 170A can be used to play audio in the handheld mode and in the hands-free mode. This speaker is also called a "two-in-one speaker".

[0072] In the embodiment of the present application, the number of the speakers 170A and / or the receivers 170B that can be used to play audio in the handheld mode is at least two. The at least two speakers 170A and / or the receivers 170B are located at different positions of the electronic device 100.

[0073] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0074] Figure 2 1 is an example of an audio system architecture diagram of an electronic device 100 according to an embodiment of the present application. It should be understood that the hierarchical architecture of the system may include several layers, each layer has a clear role and division of labor, and the layers communicate with each other through software interfaces.

[0075] In the embodiments of the present application, the audio (Audio) architecture based on the Android system is referred to as the "Android audio system architecture" or "Android audio framework", which is hereinafter referred to as the "audio system". Specifically, the audio system can realize the functions of collecting and outputting audio data, controlling audio streams, managing audio devices, and adjusting volume.

[0076] It should be understood that the electronic device 100 can play audio of different applications based on the audio system. The external playback interface of the audio system is the audio track. Each application will create a corresponding Audiotrack in the application framework layer. Each Audio track can further call the audio system engine (Audio Flinger) to achieve audio playback. Figure 2 Provide detailed explanation.

[0077] For example, Figure 2 As shown, the Android audio system architecture of the embodiment of the present application includes an application layer, an application framework layer (framework), a hardware abstraction layer and a kernel layer from top to bottom.

[0078] like Figure 2 As shown, the application layer may include a series of application packages, such as application packages corresponding to applications such as calling, instant messaging, and music. Among them, instant messaging applications may be, for example, WeChat®, QQ®, etc.

[0079] The application framework layer can provide an application programming interface (API) and a programming framework, namely, an "audio system framework", for the application programs of the application layer. The application framework layer can also include some predefined functions, classes or interfaces.

[0080] like Figure 2 As shown, in the embodiment of the present application, the application framework layer can be further divided into a Java layer and a native layer. Specifically, the Java layer can include a programming interface (application programming interface, API) of a Java class application. In other words, the Java layer can include Java APIs corresponding to a variety of different applications for controlling the playback of audio streams, such as Java APIs for call applications, Java APIs for instant messaging applications, and Java APIs for music.

[0081] The Native layer includes the Audio track, audio system engine (Audio Flinger) and audio policy service (Audio Policy Service) corresponding to each application. The Native layer can parse and compile rc files. For example, when a user answers a call, the audio system is triggered to start, specifically, an audio service (Audioserver) of the audio system is triggered to start. The file of the audio service is the audioserver.rc file. The Native layer can parse the audioserver.rc file and compile the audioserver.rc file into an audio executable program. Each application can create a corresponding Audio track in the Native layer, and the Audio track can output decoded audio data.

[0082] When the audio system is started, the audio service can load the audio system engine. During the process, the audio service can call the audio system engine and the audio policy service respectively, and initialize the audio system engine and the audio policy service.

[0083] It should be understood that the audio policy service and the audio system engine are the two basic services of the audio system. Among them, the audio policy service is the maker of the audio system policy, responsible for the policy decision of audio device switching, volume adjustment policy, etc. For example, the audio policy service can play a routing role, and it has the right to decide which audio playback device (speaker, receiver or headset, etc.) the audio stream generated by a certain Audio Track will eventually be transmitted to. For example, in an embodiment of the present application, when the audio decision service determines that the call mode is the handheld mode based on the user's choice in the call application, the decision audio playback device is the speaker and / or receiver (hereinafter referred to as the target audio playback device) used to play audio in the handheld mode, then the audio stream generated by the Audio Track corresponding to the call application is eventually transmitted to the target audio playback device.

[0084] The audio system engine is the executor of the audio playback strategy of the audio system, responsible for managing the devices of the input audio stream, managing the devices of the output audio stream, switching the audio mode, loading the audio parameters, processing and transmitting the audio stream data, etc. During the initialization of the audio system engine, the initialization default volume parameters of the audio system can be obtained, for example, the volume levels corresponding to the call stream, system stream, ring stream, warning stream (alarm), notification stream, dial key stream, etc. can be obtained.

[0085] It should be understood that during the initialization of the audio policy service, the audio configuration in the settings can be parsed, and the entire audio system is deployed using the interface provided by the audio system engine, thereby providing underlying support for the subsequent upper-layer applications to use audio devices.

[0086] It should also be understood that the audio policy service is the place where the audio system engine is actually called. The audio policy service can call the functions in the audio system engine through the binder mechanism, that is, call the audio system engine interface.

[0087] Exemplarily, in the embodiment of the present application, when an application outputs an audio stream, the audio signal included in the audio stream can be identified as a certain stream type, and the decoded audio stream data is output to the audio system engine through the Audio track corresponding to the application. The audio system engine can load the volume corresponding to the audio stream type according to the audio stream type.

[0088] In an embodiment of the present application, the audio system engine may also, based on the decision result of the audio decision service, perform sound field control and psychoacoustic masking processing on the audio signal output by the corresponding Audio track when it is determined that the output mode of the voice is the handheld mode, so as to output audio signals with different parameters (such as amplitude and phase) to each speaker and / or receiver.

[0089] For details, see Figure 2 , the audio system engine may include a masking generation module, a spectrum correction module, a sound field control module and an audio superposition module. The masking generation module is used to generate an initial masking signal according to the target audio signal. The spectrum correction module is used to perform spectrum correction on the initial masking signal to obtain a masking signal. Optionally, the spectrum correction module may be implemented by a finite impulse response (FIR) filter or an infinite impulse response (IIR) filter.

[0090] The sound field control module is used to perform sound field control on the target audio signal and the masking signal respectively. The target audio signal refers to the audio signal that needs to be output by the target audio playback device. Optionally, the target audio signal can be a call downlink signal output by a call application, a voice audio signal output by an instant messaging application, or an audio signal output by other applications, such as a recording audio signal output by a recording application. The embodiments of the present application do not impose any restrictions on the specific source of the target audio signal.

[0091] The sound field control module is used to control the sound field of the target audio signal and the masking signal in different regions based on the distribution position of the target audio playback device. Specifically, the sound field control module may include a first filter group and a second filter group. The first filter group includes N first control filters. The second filter group includes N second control filters. Where N is Figure 1The number of target audio playback devices shown in , N is an integer greater than or equal to 2. The N first control filters correspond one-to-one to the N second control filters, and correspond one-to-one to the N target audio playback devices. The N first control filters are used to process the target audio signals respectively, output N target output signals with different parameters, and realize the sound field control of the target audio signals. The N control filters in the second filter group are used to process the masking signals respectively, output N masking output signals with different parameters, and realize the sound field control of the masking signals.

[0092] The audio superposition module is used to superimpose the target audio signal output by each first control filter with the masked output signal output by the corresponding second control filter to obtain a signal to be played. Each signal to be played is finally played by the corresponding target audio playback device.

[0093] It can be understood that in the embodiments of the present application, the spectrum correction module, the sound field control module, the audio superposition module, etc. can be software modules or modules that combine software and hardware, and the embodiments of the present application do not impose any limitations on this.

[0094] like Figure 2 As shown, in the embodiment of the present application, the hardware abstraction layer may include an audio hardware abstraction layer (Audio hardware abstraction layer, Audio HAL). The audio hardware abstraction layer may be responsible for interaction with audio hardware devices, and the audio hardware abstraction layer may be directly called by the audio system engine.

[0095] The audio hardware abstraction layer can select an audio hardware device for the audio stream of this type and access the driver of the hardware device in the kernel layer, thereby driving the audio hardware device to play according to the volume corresponding to the audio stream type loaded by the audio system engine.

[0096] The kernel layer is a layer between hardware and software. The kernel layer includes at least audio drivers, etc. Specifically, the audio drivers may include earpiece drivers, speaker drivers, headphone interface drivers, etc. Among them, the earpiece driver may include one or more. When the earpiece driver includes one, the one earpiece driver may include one driver to drive at least two earpieces. When the earpiece driver includes multiple, one earpiece driver may be responsible for driving one earpiece. The embodiments of the present application do not impose any limitation on this.

[0097] In addition, the audio system may also include an audio hardware device of the electronic device 100, which may be called by a corresponding driver of the kernel layer. The speaker, receiver and other hardware devices of the mobile phone play based on the volume loaded by the audio system engine.

[0098] The audio hardware device of the electronic device 100 may include Figure 1 The speaker 170A and / or earpiece (receiver 170B), a wired headset connected through the headphone interface 170D, and other playback devices capable of playing audio are not limited to this embodiment of the present application.

[0099] For ease of understanding, the following embodiments of the present application will be described in detail with Figure 1 and Figure 2 Taking the electronic device of the structure shown as an example, combined with the accompanying drawings, taking the call scenario as an example, the audio signal processing method provided in the embodiment of the present application is specifically described.

[0100] Figure 3 is a flowchart of an example of an audio signal processing method provided in an embodiment of the present application. Figure 4 This is a schematic diagram of the principle of an audio signal processing method provided in an embodiment of the present application. Please refer to Figure 3 and Figure 4 , the method comprising:

[0101] S101. A call application outputs a call downlink signal to a sound field control module and a masking generation module in an audio system engine.

[0102] The call downlink signal is the audio signal transmitted by the opposite device received by the call application. The call application creates a corresponding Audio track in the application framework layer. During the call, the Audio track corresponding to the call application calls the audio system engine and outputs the call downlink signal to the audio system engine. Specifically, the call application can output the call downlink signal to the sound field control module and the masking generation module in the audio system engine respectively.

[0103] It can be understood that the call downlink signal is an audio stream, and the call application can output the signal to the sound field control module frame by frame. In a specific embodiment, the frame length of the call downlink signal can be, for example, 1 ms or 20 ms.

[0104] S102: The first filter group in the sound field control module performs sound field control on the call downlink signal to obtain N downlink output signals.

[0105] It can be understood that when the user uses the target audio playback device to make a call, the sound field can be divided into the in-ear sound field and the leakage sound field. The in-ear sound field is also called the ear canal sound field, the target sound field, etc., which refers to the sound field in the user's ear canal and the surrounding preset range. The leakage sound field is also called the extra-ear sound field, which refers to the sound field outside the in-ear sound field (the side away from the target audio playback device). In the embodiment of the present application, the sound in the in-ear sound field is the sound that needs to be heard by the user, that is, the sound that the user is interested in, that is, the intelligibility of the sound in the in-ear sound field should be as high as possible. The sound in the leakage sound field is leakage sound, which is a sound that is not expected to be heard by people around, that is, the intelligibility of the sound in the leakage sound field should be as low as possible.

[0106] Based on this, in an embodiment of the present application, the area around the electronic device is pre-divided into an in-ear area (also referred to as a first spatial area) and a sound leakage area (also referred to as a second spatial area). The in-ear area is the area corresponding to the in-ear sound field, that is, the area within a preset range around the target audio playback device of the electronic device. The sound leakage area is the area corresponding to the sound leakage sound field, that is, the area outside the in-ear area (away from the side of the target audio playback device). It can be understood that the specific ranges of the in-ear area and the sound leakage area can be divided according to actual needs.

[0107] The sound field control module performs regional sound field control on the in-ear area and the sound leakage area. In this step, the first filter group takes the in-ear area as the bright area and the sound leakage area as the dark area, performs sound field control test or simulation, and maximizes the energy ratio of the bright area and the dark area to solve the filter coefficient to generate the filter group. The first filter group can control the sound field of the call downlink signal, so that the in-ear area can generate a call downlink signal sound field with larger energy as much as possible, and the sound leakage area can generate a call downlink signal sound field with smaller energy as much as possible. In other words, the first filter group controls the sound field of the call downlink signal, so that the energy ratio of the in-ear sound field formed by the call downlink signal in the in-ear area and the sound leakage sound field formed in the sound leakage area is as large as possible. In this way, the intelligibility and clarity of the call sound heard by the user are high, while the intelligibility and clarity of the call sound heard by the surrounding people are low, which effectively prevents sound leakage and protects the privacy of the user's call.

[0108] For details, see Figure 4, the sound field control module includes a first filter group and a second filter group. The first filter group includes N first control filters: first control filter 1, first control filter 2...first control filter N. The second filter group includes N second control filters: second control filter 1, second control filter 2...second control filter N. Among them, the first control filter 1 corresponds to the second control filter 1, which is the control filter corresponding to the target audio playback device 1, and is used to control the amplitude and phase of the audio signal input to the target audio playback device 1. The first control filter 2 corresponds to the second control filter 2, which is the control filter corresponding to the target audio playback device 2, and is used to control the amplitude and phase of the audio signal input to the target audio playback device 2...The first control filter N corresponds to the second control filter N, which is the control filter corresponding to the target audio playback device N, and is used to control the amplitude and phase of the audio signal input to the target audio playback device N.

[0109] See also Figure 4 The call application can input the call downlink signal into each first control filter in the first filter group respectively, wherein the signal obtained after the first control filter 1 processes the call downlink signal is called downlink output signal 1, the signal obtained after the first control filter 2 processes the call downlink signal is called downlink output signal 2... The signal obtained after the first control filter N processes the call downlink signal is called downlink output signal N.

[0110] That is to say, the N first control filters in the first filter group are used to perform sound field control on the call downlink signal, so that the parameters (including amplitude, phase, etc.) of the call downlink signal ultimately output by different target audio playback devices are different, thereby making the energy ratio of the in-ear sound field and the leakage sound field of the call downlink signal as large as possible.

[0111] S103: The sound field control module outputs N downstream output signals to the audio superposition modules respectively.

[0112] S104: The masking generation module generates an initial masking signal according to the call downlink signal, and the frequency spectrum of the initial masking signal is consistent with the frequency spectrum of the call downlink signal.

[0113] Optionally, the masking generation module can obtain a preset signal from a preset masking signal library, and adjust the spectrum of the preset signal according to the call downlink signal to obtain an initial masking signal. Optionally, the preset signal in the preset masking signal library can be, for example, white noise, music, or ocean wave sound, etc., and the present application embodiment does not impose any limitation on this.

[0114] Optionally, the masking generation module may also generate an initial masking signal after processing the call downlink signal, which will be described in further detail in subsequent embodiments.

[0115] It can be understood that in the embodiment of the present application, the spectrum consistency may mean that the spectrum is completely identical, or the spectrum correlation may be higher than a preset threshold.

[0116] S105: The mask generation module outputs the initial mask signal to the spectrum correction module.

[0117] S106: The spectrum correction module performs spectrum correction on the initial masking signal to obtain a masking signal.

[0118] S107: The spectrum correction module outputs the masking signal to the sound field control module.

[0119] S108 . The second filter group in the sound field control module performs sound field control on the masking signal to obtain N masking output signals.

[0120] In this step, the second filter group uses the sound leakage area as the bright area and the ear area as the dark area to perform sound field control test or simulation, maximize the sound field energy ratio of the bright area and the dark area, and solve the filter generated by the filter coefficient. That is to say, when the second filter group controls the sound field, the bright and dark areas corresponding to the first filter group are swapped. In this way, the second filter group performs sound field control on the masking signal, which can make the ear area produce a masking signal sound field with less energy as much as possible, and make the sound leakage area produce a masking signal sound field with greater energy as much as possible. That is, the second filter group controls the sound field of the masking signal so that the energy ratio of the sound leakage sound field of the masking signal to the sound field in the ear is as large as possible. In this way, the final masking signal has a smaller masking force on the call downlink signal in the ear area, which does not affect the intelligibility and clarity of the call sound heard by the user, while the masking signal has a greater masking force on the call downlink signal in the sound leakage area, effectively reducing the intelligibility and clarity of the sound in the sound leakage area, thereby effectively preventing sound leakage and protecting the privacy of user calls.

[0121] For details, see Figure 4 The spectrum correction module can input the masking signal into each second control filter in the second filter group respectively, wherein the signal obtained after the second control filter 1 processes the masking signal is called masking output signal 1, the signal obtained after the second control filter 2 processes the masking signal is called masking output signal 2... The signal obtained after the second control filter N processes the masking signal is called masking output signal N.

[0122] That is to say, the N second control filters in the second filter group are used to perform sound field control on the masking signal, so that the parameters (including amplitude, phase, etc.) of the masking signal ultimately output by different target audio playback devices are different, thereby making the energy ratio between the leakage sound field formed by the masking signal in the leakage area and the in-ear sound field formed by the masking signal in the in-ear area as large as possible.

[0123] S109: The sound field control module outputs the N-channel masking output signals to the audio superposition modules respectively.

[0124] S110 , the audio superposition module superimposes each downlink output signal with the corresponding masking output signal to obtain a to-be-played signal corresponding to each target audio playback device.

[0125] As described above, the N first control filters correspond to the N second control filters in a one-to-one manner. Therefore, the N downlink output signals correspond to the N masked output signals in a one-to-one manner.

[0126] For details, see Figure 4 The audio superposition module superimposes the downlink output signal 1 with the shielded output signal 1 to obtain the to-be-played signal 1 corresponding to the target audio playback device 1; superimposes the downlink output signal 2 with the shielded output signal 2 to obtain the to-be-played signal 2 corresponding to the target audio playback device 2… The downlink output signal N is superimposed with the shielded output signal N to obtain the to-be-played signal N corresponding to the target audio playback device N.

[0127] It can be understood that the sound field control module can process the call downlink signal and the masking signal frame by frame, while the audio superposition module can superimpose multiple frames of signals at a time. Specifically, the frame length of the masking signal is generally larger than the call downlink signal, and can be n times the call downlink signal, where n is a positive integer. Therefore, the frame length of the masking output signal is n times the frame length of the downlink output signal. When superimposing signals, the audio superposition module superimposes the downlink output signal corresponding to the n-frame call downlink signal with the masking output signal corresponding to the n-frame call downlink signal. For example, the downlink output signal corresponding to the call downlink signal from the 1st to the 50th frame is superimposed with the masking output signal corresponding to the call downlink signal from the 1st to the 50th frame to obtain a signal to be played.

[0128] As described above, the first filter group makes the energy ratio of the in-ear sound field formed by the call downlink signal in the in-ear area to the leakage sound field formed in the sound leakage area as large as possible, that is, the energy of the in-ear sound field formed by the call downlink signal is greater than the energy of the leakage sound field formed by the call downlink signal. The second filter group makes the energy ratio of the leakage sound field formed by the masking signal in the sound leakage area to the in-ear sound field formed by the masking signal in the in-ear area as large as possible, that is, the energy of the in-ear sound field formed by the masking signal is less than the energy of the leakage sound field formed by the masking signal. Therefore, the signal to be played after superimposing the call downlink signal and the masking signal forms a sound field in the in-ear area that is much larger than the energy of the sound field formed in the sound leakage area. In this way, the user's hearing sense in the ear canal can be guaranteed, the intelligibility of the leakage sound can be reduced, and the user's privacy can be protected.

[0129] S111, the audio superposition module outputs each channel of the to-be-played signal to the audio hardware abstraction layer.

[0130] S112: The audio hardware abstraction layer accesses the driver of the target audio playback device corresponding to the kernel layer, thereby driving each target audio playback device to play the corresponding signal to be played.

[0131] For details, see Figure 4 , target audio playback device 1 plays signal 1 to be played; target audio playback device 2 plays signal 2 to be played...target audio playback device N plays signal N to be played.

[0132] In summary, the audio signal processing method provided in the embodiment of the present application, on the one hand, controls the sound field of the call downlink signal through the sound field control module, so that the energy ratio of the in-ear sound field of the call downlink signal and the leakage sound field of the call downlink signal is as large as possible. In this way, sound leakage is reduced, user privacy is protected, and user experience is improved. On the other hand, a masking signal is generated, and the sound field of the masking signal is controlled by the sound field control module, so that the energy ratio of the leakage sound field of the masking signal and the in-ear sound field of the masking signal is as large as possible. In this way, a smaller degree of psychoacoustic masking is performed on the call downlink signal in the in-ear area, and a larger degree of psychological masking is performed on the call downlink signal in the leakage area, thereby reducing the intelligibility of the sound in the leakage area while minimizing the impact on the in-ear area as much as possible, that is, while protecting the user's privacy, reducing the impact on the user's hearing sense and improving the user experience. Thirdly, since the second control filter will introduce spectral distortion during sound field control, in this embodiment, the initial masking signal is spectrally corrected to generate a masking signal, and then the masking signal is input into the sound field control module. This can correct the spectral distortion during sound field control, improve the masking effect, and further improve the final privacy protection effect.

[0133] Optionally, in some embodiments, the spectrum correction of the initial masking signal may also be performed after the sound field control, which is not limited in any way in the embodiments of the present application.

[0134] It can be understood that the order in which the electronic device executes the above steps is not limited. For example, in a specific embodiment, the above step S102 and the above steps S103 to S108 can be executed simultaneously.

[0135] The generation process of the initial masking signal is described below.

[0136] For example, Figure 5 FIG. 1 is a flow chart of another example of a method for processing an audio signal provided in an embodiment of the present application. Figure 5 As shown, the above step "S104, the masking generation module generates an initial masking signal according to the call downlink signal" may include the following steps. The following steps are performed by the masking generation module and will not be described in detail.

[0137] S201, buffering a call downlink signal having a duration equal to a preset masking frame length, and obtaining a current frame signal to be masked.

[0138] The preset masking frame length is also the frame length of the masking signal finally generated. Optionally, the preset masking frame length may be greater than 50 ms and less than 300 ms, so that the masking signal can have a better masking effect.

[0139] Specifically, the masking generation module starts from receiving the call downlink signal output by the call application, and caches the call downlink signal according to the preset masking frame length. Each cached call downlink signal with a duration of the preset masking frame length is called a frame of signal to be masked. For the sake of distinction, in the embodiment of the present application, the frame of signal to be masked obtained by the most recent cache at the current moment is called the current frame of signal to be masked. The most recent frame of signal to be masked before the current frame of signal to be masked is called the previous frame of signal to be masked (or the previous frame of signal to be masked).

[0140] As described above, the call application can output the call downlink signal to the mask generation module frame by frame, and the frame length of the call downlink signal can be, for example, 1ms, 20ms, etc. Taking the frame length of the call downlink signal as 1ms and the preset mask frame length as 50ms as an example, the mask generation module caches the call downlink signal according to the preset mask frame length, and obtains a call downlink signal of 50ms in length after caching 50ms (i.e., 50 frames) of the call downlink signal. The 50ms call downlink signal is called a frame of signal to be masked.

[0141] It can be understood that the current frame signal to be masked, that is, the signal simultaneously input into the first filter group in the sound field control module, is actually the signal that needs to be masked by the final masking signal, that is, the target audio signal, also called the current frame signal to be masked.

[0142] S202 , performing windowing processing on the signal to be masked in the current frame and then performing time inversion to obtain an inverted signal of the current frame.

[0143] Optionally, a Hamming window, a rectangular window, a Hanning window, etc. may be used for windowing, and the embodiments of the present application do not impose any limitation on this.

[0144] The signal obtained after windowing and time reversing the signal to be masked is called the inversion signal. For the sake of distinction, the signal obtained after windowing and time reversing the signal to be masked in the current frame is called the inversion signal of the current frame, and the signal obtained after windowing and time reversing the signal to be masked in the previous frame is called the inversion signal of the previous frame (or the inversion signal of the previous frame).

[0145] S203 , superimposing the current frame inversion signal with the previous frame inversion signal to obtain a current frame basic mask signal.

[0146] In this step, the current frame inversion signal is superimposed on the previous frame inversion signal, which can improve the smoothness of the obtained basic masking signal, and further improve the smoothness of the finally obtained masking signal.

[0147] Optionally, when the current frame inversion signal is the first frame inversion signal, the previous frame inversion signal may be assumed to be 0, and the current frame basic mask signal is obtained by superposition.

[0148] S204: According to a preset TMR threshold (also called a preset energy ratio threshold) and the energy of the signal to be masked in the current frame, adjust the energy of the basic masking signal of the current frame to obtain an initial masking signal of the current frame.

[0149] Optionally, the preset TMR threshold may be, for example, 0 dB.

[0150] It can be understood that the masking effect of the masking signal on the target audio signal is related to the TMR. By adjusting the energy of the basic masking signal so that the energy ratio of the current frame to be masked signal and the basic masking signal is a preset threshold, the masking effect of the masking signal can be improved, thereby improving the effect of call privacy protection.

[0151] By repeatedly executing the above steps S201 to S204, it is possible to continuously generate the corresponding initial masking signal according to the call downlink signal.

[0152] In this embodiment, the call downlink signal is processed in real time to obtain an initial masking signal, so that the spectrum of the final masking signal is more consistent with the call downlink signal, thereby improving the masking effect of the masking signal and thus improving the effect of call privacy protection.

[0153] As described above, when the preset masking frame length is large, the masking effect of the masking signal can be improved. However, in actual communication, a large preset masking frame length may cause a time delay (time difference) between the masking signal and the call downlink signal, resulting in the generated masking signal not corresponding to the actual call downlink signal spectrum, resulting in a time delay, which in turn causes the sound leakage to not be masked, resulting in invalid masking; or causing the call downlink signal that the user expects to hear to be over-masked, reducing the user's hearing sense, thereby affecting the user experience. Based on this, in an embodiment of the present application, when generating an initial masking signal, it can be combined with sound energy detection to generate an effective and accurate initial masking signal to improve the masking effect of the final masking signal. The following is an explanation in conjunction with the accompanying drawings.

[0154] For example, Figure 6 FIG. 1 is a flowchart of another method for processing an audio signal provided in an embodiment of the present application. Figure 6 As shown, the above step "S204, adjusting the energy of the basic masking signal of the current frame according to the preset TMR threshold and the energy of the signal to be masked in the current frame to obtain the initial masking signal of the current frame" includes:

[0155] S301. Adjust the energy of the basic masking signal of the current frame to obtain the masking adjustment signal of the current frame, wherein the energy ratio of the signal to be masked of the current frame to the masking adjustment signal of the current frame is equal to a preset TMR threshold.

[0156] That is to say, the energy of the current frame basic masking signal is adjusted so that the energy ratio of the current frame to-be-masked signal to the current frame basic masking signal is equal to the preset TMR threshold, and the adjusted current frame basic masking signal is called the current frame masking adjustment signal.

[0157] S302. Determine whether there is a valid signal in the signal to be masked in the current frame according to the call energy threshold; if not, execute step S303; if yes, execute step S304.

[0158] Optionally, the call energy threshold may be a preset value, or may be determined by a relevant module in the electronic device based on voice activity detection, and the embodiments of the present application do not impose any limitation on this.

[0159] Determine whether the energy of the signal to be masked in the current frame is greater than or equal to the call energy threshold. If so, determine that there is a valid signal in the signal to be masked in the current frame; if not, determine that there is no valid signal in the signal to be masked in the current frame.

[0160] It can be understood that the existence of a valid signal in a certain frame of sound signal indicates that the frame of sound signal is a signal with a relatively loud sound; the absence of a valid signal indicates that the frame of signal is a signal with no sound or a relatively low sound. The existence of a valid signal in the current frame of the signal to be masked indicates that the frame signal to be masked is a signal with a relatively loud sound and the frame signal to be masked needs to be masked; the absence of a valid signal indicates that the frame signal to be masked is a signal with no sound or a relatively low sound and the frame signal to be masked does not need to be masked or requires a relatively low degree of masking. It can be understood that if the call downlink signal without a valid signal is normally masked, the intelligibility of the user during the call will be reduced and the user's hearing will be affected.

[0161] S303: reduce the amplitude of the current frame mask adjustment signal to obtain the current frame initial mask signal, or delete the current frame mask adjustment signal and set the current frame initial mask signal to null or 0.

[0162] If there is no valid signal in the signal to be masked in the current frame, execute this step. If there is no valid signal in the current frame signal to be masked, it means that the frame signal to be masked is a signal without sound or with a relatively low sound, and the frame signal to be masked does not need to be masked or requires a relatively small degree of masking. Therefore, the amplitude of the current frame masking adjustment signal is reduced to obtain the initial masking signal of the current frame, and the masking signal obtained after the spectrum correction of the initial masking signal of the current frame is used to mask the frame signal to a relatively small extent. Alternatively, the current frame masking adjustment signal is deleted, and the initial masking signal of the current frame is set to empty or 0, and the frame signal to be masked is not masked subsequently. In this way, excessive masking of the call downlink signal of the signal without sound or with a relatively low sound is prevented, thereby preventing the user's hearing sense from being affected.

[0163] S304, determining whether there is a valid signal in the current frame mask adjustment signal according to the mask energy threshold; if not, executing step S305; if yes, no processing is performed.

[0164] Optionally, the masking energy threshold can be determined based on the call energy threshold and the energy adjustment value in step S301. Specifically, the masking energy threshold can be the sum of the call energy threshold and the energy adjustment value. The energy adjustment value is the energy difference between the current frame masking adjustment signal and the current frame basic masking signal. The energy adjustment value has positive and negative values. In step S301, if the energy of the current frame basic masking signal is increased (that is, the energy of the current frame masking adjustment signal > the energy of the current frame basic masking signal), the energy adjustment value is a positive value; if the energy of the current frame basic masking signal is decreased (that is, the energy of the current frame masking adjustment signal < the energy of the current frame basic masking signal), the energy adjustment value is a negative value.

[0165] It is determined whether the energy of the current frame mask adjustment signal is greater than or equal to the mask energy threshold. If so, it is determined that there is a valid signal in the current frame mask adjustment signal; if not, it is determined that there is no valid signal in the current frame mask adjustment signal.

[0166] S305: Obtain a preset signal from a preset masking signal library, and adjust the frequency spectrum of the preset signal according to the frequency spectrum of the signal to be masked in the current frame to obtain an initial masking signal in the current frame.

[0167] That is to say, if there is a valid signal in the signal to be masked of the current frame, and there is no valid signal in the mask adjustment signal of the current frame, then this step is executed. If there is no valid signal in the mask adjustment signal of the current frame, it means that the signal is a silent signal or a signal with a relatively low sound, and the signal cannot effectively mask the frame signal to be masked. Therefore, the preset signal is directly obtained from the preset mask signal library, and the spectrum of the preset signal is adjusted to be consistent with the spectrum of the signal to be masked of the current frame, so as to obtain the initial mask signal of the current frame. In this way, the occurrence of invalid masking phenomenon can be prevented and the masking effect can be improved.

[0168] It can be understood that if the initial masking signal is not generated according to the process shown in the above steps S201 to S204, but is directly generated based on the preset signal in the preset masking signal library, in this case, the sound energy detection can also be performed according to the process similar to the above steps S301 to S304 to generate an effective and accurate initial masking signal to improve the masking effect of the masking signal. The difference is that in this case, the signals generated according to the preset signal all have valid signals, so there is no situation of step S305, and step S305 is not executed.

[0169] The generation process of the sound field control module is described below.

[0170] As described above, the in-ear area and the sound leakage area can be pre-divided. When generating the sound field control module, a preset number of sound field sampling points can be set in the in-ear area and the sound leakage area respectively, and the sound field control is simulated or tested based on the sound field sampling points, so as to find the optimal solution filter coefficient, obtain the filter group, and then obtain the sound field control module. Optionally, the filter coefficient can be solved by the sound energy contrast method, the sound pressure matching method, etc. to obtain the filter group, or the filter coefficient can be solved by the fusion algorithm of the sound energy contrast method and the sound pressure matching method to obtain the filter group. This embodiment mainly takes the sound energy contrast method as an example for explanation.

[0171] For example, Figure 7 This is a schematic diagram of the distribution of sound field sampling points provided in an embodiment of the present application. Figure 7 As shown, the sampling points in the ear area are shown as black dots. When the user holds an electronic device to answer a call, the sampling points are points within the preset range of the user's ear canal and the ear. The sampling points in the sound leakage area are shown as triangles. The sampling points in the sound leakage area are distributed outside the ear area (i.e., away from the electronic device).

[0172] In one embodiment, Figure 7 Taking the sound field sampling points shown in the figure as an example, the first filter group and the second filter group can be solved and generated according to the following process:

[0173] 1) Solution generation of the first filter bank

[0174] The sampling points in the ear area are taken as the bright area, and the sampling points in the sound leakage area are taken as the dark area. The energy ratio between the bright area and the dark area is maximized, and the optimal filter coefficient is solved to obtain the first filter group. The specific solution process includes:

[0175] a. Set the target frequency band of the first filter group.

[0176] The target frequency band of the first filter group refers to the frequency range that the first filter can process, for example, the target frequency band may be f1 to f2, f1 may be, for example, 100 Hz, and f2 may be, for example, 4 kHz.

[0177] b. Determine the transfer function matrix of the audio signal in the target frequency band in the bright and dark areas.

[0178] Optionally, the transfer function matrix (referred to as the bright area transfer function matrix) of the audio signal in the target frequency band transmitted from each target audio playback device to the NB sampling points in the bright area can be determined by testing or simulation. ); determine the transfer function matrix (called dark area transfer function matrix) of the audio signal in the target frequency band from each target audio playback device to the ND sampling points in the dark area ).

[0179] Wherein, NB is the number of sampling points in the bright area, and ND is the number of sampling points in the dark area. In this embodiment, the number of sampling points in the bright area is also the number of sampling points in the ear area, and the number of sampling points in the dark area is also the number of sampling points in the sound leakage area. Figure 6 Taking the sound field sampling points shown as an example, NB is 3 and ND is 12.

[0180] Then, the bright area transfer function matrix The size is NFFTxNBxN, the dark area transfer function matrix The size is NFFTxNDxN, where NFFT is the number of Fourier transform frequency points in the target frequency band, and N is the number of target audio playback devices.

[0181] c. Determine the spatial correlation matrix of the bright area and the dark area based on the transfer function matrix of the bright area and the dark area.

[0182] Specifically, according to the bright area transfer function matrix , based on formula (1), determine the bright area spatial correlation matrix :

[0183] (1)

[0184] Here, H represents the conjugate transpose.

[0185] According to the dark area transfer function matrix , based on formula (2), determine the bright area spatial correlation matrix :

[0186] (2)

[0187] d. Solve the first filter group based on the spatial correlation matrix of the bright area and the dark area.

[0188] Specifically, the first filter group Q is expressed as formula (3):

[0189] (3)

[0190] It can be understood that the size of the first filter group Q is NFFTxN. Among them, the first control filter corresponding to the i-th target audio playback device is .

[0191] 2) Solution generation of the second filter bank

[0192] The sampling points in the sound leakage area are taken as the bright area, and the sampling points in the ear area are taken as the dark area. The energy ratio between the bright area and the dark area is maximized, and the optimal filter coefficient is solved to obtain the second filter group. In other words, the bright area and the dark area are swapped when the first filter group and the second filter group are solved and generated.

[0193] For example, Figure 8 The corresponding relationship between the first filter group and the second filter group in the sound field when solving the generation is shown. Figure 8 As shown, the sampling points in the ear area are taken as the bright area, and the sampling points in the sound leakage area are taken as the dark area for sound field control, and the filter coefficients are solved to obtain the first filter group. The sampling points in the sound leakage area are taken as the bright area, and the sampling points in the ear area are taken as the dark area for sound field control, and the filter coefficients are solved to obtain the second filter group.

[0194] In this way, the sound field control of the target audio signal (such as the call downlink signal) is subsequently performed through the first filter group, so that the energy of the in-ear sound field of the target audio signal is larger, and the energy of the leakage sound field of the target audio signal is smaller; the sound field control of the masking signal is performed through the second filter group, so that the energy of the in-ear sound field of the masking signal is smaller, and the energy of the leakage sound field of the masking signal is larger. After the target audio signal and the masking signal are superimposed, the intelligibility of the in-ear sound field of the final output sound is not affected, while the intelligibility of the leakage sound field is reduced, thereby effectively protecting the user's privacy and improving the user experience without affecting the user's hearing sense.

[0195] The process of solving the second filter group refers to the above steps a to d, which will not be described in detail.

[0196] The above describes in detail an example of a method for processing an audio signal provided by an embodiment of the present application. It is understandable that, in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to exceed the scope of the present application.

[0197] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each function can be divided into various functional modules, such as a detection unit, a processing unit, a display unit, etc., or two or more functions can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0198] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0199] The electronic device provided in this embodiment is used to execute the above-mentioned audio signal processing method, and thus can achieve the same effect as the above-mentioned implementation method.

[0200] In the case of an integrated unit, the electronic device may further include a processing module, a storage module and a communication module. The processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device to execute stored program codes and data, etc. The communication module may be used to support the electronic device to communicate with other devices.

[0201] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0202] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a Figure 1 Device of the structure shown.

[0203] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the audio signal processing method of any of the above embodiments.

[0204] The embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the audio signal processing method in the above-mentioned embodiment.

[0205] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, a component or a module, and the device may include a connected processor and a memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the audio signal processing method in the above-mentioned method embodiments.

[0206] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0207] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0208] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0209] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0210] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0211] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0212] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for processing an audio signal, applied to an electronic device, wherein the electronic device comprises N target audio playback devices, where N is an integer greater than or equal to 2, characterized in that: The method comprises: Acquire a target audio signal, where the target audio signal is an audio signal that needs to be played by the N target audio playing devices; generating a masking signal according to the target audio signal; Performing a first sound field control process on the target audio signal to obtain N target output signals corresponding one-to-one to the N target audio playback devices, wherein the energy of the sound field formed by the target output signal in a first spatial region is greater than the energy of the sound field formed in a second spatial region, wherein the first spatial region is closer to the N target audio playback devices than the second spatial region; Performing a second sound field control process on the masking signal to obtain N masking output signals corresponding to the N target audio playback devices, wherein the energy of the sound field formed by the masking output signal in the second spatial area is greater than the energy of the sound field formed in the first spatial area; Superimposing the target output signal corresponding to each of the target audio playback devices with the corresponding masked output signal to obtain N channels of to-be-played signals corresponding one by one to the N target audio playback devices; The corresponding signals to be played are played respectively through the N target audio playing devices.

2. The method according to claim 1, characterized in that: The first spatial area is a spatial area within a preset range around the N target audio playback devices, the second spatial area does not overlap with the first spatial area, and the second spatial area is located on a side of the first spatial area away from the electronic device.

3. The method according to claim 1, characterized in that The electronic device comprises a first filter group and a second filter group; The performing the first sound field control processing on the target audio signal to obtain N target output signals corresponding to the N target audio playback devices one by one, comprises: Performing a first sound field control process on the target audio signal through the first filter group to obtain the N target output signals; The performing a second sound field control process on the masking signal to obtain N masking output signals corresponding to the N target audio playback devices one by one, comprises: The second sound field control processing is performed on the masking signal through the second filter group to obtain the N-channel masking output signals.

4. The method according to claim 3, characterized in that The first filter group is a filter group generated by taking the first spatial area as a bright area and the second spatial area as a dark area, performing a sound field control test or simulation, maximizing the sound field energy ratio between the bright area and the dark area, and solving the filter coefficients; The second filter group is a filter group generated by taking the second spatial area as a bright area and the first spatial area as a dark area, performing sound field control test or simulation, maximizing the sound field energy ratio between the bright area and the dark area and solving the filter coefficients.

5. The method according to any one of claims 1 to 4, characterized in that The masking signal is consistent with the frequency spectrum of the target audio signal.

6. The method according to claim 5, characterized in that Generating a masking signal according to the target audio signal comprises: generating, according to the target audio signal, an initial masking signal consistent with the frequency spectrum of the target audio signal; Performing spectrum correction on the initial masking signal to obtain the masking signal.

7. The method according to claim 6, characterized in that The step of generating, according to the target audio signal, an initial masking signal consistent with the frequency spectrum of the target audio signal comprises: Buffering the target audio signal for a time length equal to a preset masking frame length, and obtaining a signal to be masked in a current frame; Performing windowing processing on the signal to be masked in the current frame and then performing time reversal to obtain a current frame inversion signal; Acquire a previous frame inversion signal, where the previous frame inversion signal refers to a frame inversion signal that is closest to the current frame inversion signal; Superimposing the current frame inversion signal with the previous frame inversion signal to obtain a current frame basic mask signal; According to a preset energy ratio threshold and the energy of the signal to be masked in the current frame, the energy of the basic masked signal in the current frame is adjusted to obtain an initial masked signal in the current frame.

8. The method according to claim 7, characterized in that The step of adjusting the energy of the basic masking signal of the current frame according to the preset energy ratio threshold and the energy of the signal to be masked of the current frame to obtain the initial masking signal of the current frame includes: Adjusting the energy of the current frame basic masking signal to obtain a current frame masking adjustment signal, wherein an energy ratio of the current frame to-be-masked signal to the current frame masking adjustment signal is equal to the preset energy ratio threshold; respectively determining whether there are valid signals in the current frame to-be-masked signal and the current frame mask adjustment signal; If there is no valid signal in the signal to be masked of the current frame, then reducing the amplitude of the mask adjustment signal of the current frame to obtain the initial mask signal of the current frame, or taking 0 or a null value as the initial mask signal of the current frame; If there is a valid signal in the signal to be masked of the current frame and there is no valid signal in the mask adjustment signal of the current frame, a preset signal is obtained from a preset mask signal library, and the spectrum of the preset signal is adjusted according to the spectrum of the signal to be masked of the current frame to obtain the initial mask signal of the current frame.

9. The method according to claim 8, characterized in that The respectively determining whether there are valid signals in the current frame to-be-masked signal and the current frame mask adjustment signal includes: Determine whether the energy of the signal to be masked in the current frame is greater than or equal to a first energy threshold; If the energy of the signal to be masked in the current frame is greater than or equal to the first energy threshold, determining that there is a valid signal in the signal to be masked in the current frame; If the energy of the signal to be masked in the current frame is less than the first energy threshold, determining that there is no valid signal in the signal to be masked in the current frame; Determining whether the energy of the current frame mask adjustment signal is greater than or equal to a second energy threshold; If the energy of the current frame mask adjustment signal is greater than or equal to the second energy threshold, determining that there is a valid signal in the current frame mask adjustment signal; If the energy of the current frame mask adjustment signal is less than the second energy threshold, it is determined that no valid signal exists in the current frame mask adjustment signal.

10. The method according to claim 5, characterized in that Generating a masking signal according to the target audio signal comprises: Obtaining a preset signal from a preset masking signal library; The frequency spectrum of the preset signal is adjusted according to the frequency spectrum of the target audio signal to obtain the masking signal.

11. An electronic device, characterized in that: include: Processors, memory, and interfaces; The processor, the memory and the interface cooperate with each other so that the electronic device executes the method as claimed in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor calls instructions so that the electronic device executes the method according to any one of claims 1 to 10.