Audio signal processing method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to an audio signal processing method and device, electronic equipment and storage medium. The method comprises: detecting that an audio acquisition device of the electronic equipment is in a working state; generating a first audio signal based on an acquired audio signal of the audio acquisition device, the first audio signal being offset from the acquired audio signal. The method of the present disclosure can reduce the risk of user information leakage.

Description

Technical Field

[0001] This disclosure relates to the field of information security technology, and in particular to an audio signal processing method, apparatus, electronic device and storage medium. Background Technology

[0002] As people's lives become more diverse, the demand for various applications is also increasing, leading to a growing variety of applications installed on electronic devices. However, the functionality of these applications varies greatly, and some may even contain listening programs that monitor the audio in the user's environment, causing concerns about information leakage. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides an audio signal processing method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of the present disclosure, an audio signal processing method is provided, the method comprising:

[0005] The audio acquisition device of the electronic device is detected to be in working condition;

[0006] Based on the audio signal acquired by the audio acquisition device, a first audio signal is generated, and the first audio signal cancels out the acquired audio signal.

[0007] In some embodiments, the audio acquisition device of the detection electronic device is in an operational state, including:

[0008] Detect whether a first-running voice program exists;

[0009] In the presence of the first running voice program, the audio acquisition device of the electronic device is detected to be in working condition.

[0010] In some embodiments, the method further includes:

[0011] The first audio signal and the signal obtained by canceling the acquired audio signal are stored, and the stored signal is used to be acquired by the first running voice program.

[0012] In some embodiments, the method further includes:

[0013] Based on a preset signal generation strategy, a second audio signal is generated. The second audio signal is used to superimpose the signal obtained by canceling out the first audio signal and the acquired audio signal. The second audio signal is different from the acquired audio signal.

[0014] In some embodiments, the method further includes:

[0015] The first audio signal and the acquired audio signal after cancellation are filtered to obtain the target audio signal.

[0016] In some embodiments, the acquired audio signal includes an audio signal from an audio acquisition device, and the generation of a first audio signal based on the acquired audio signal from the audio acquisition device includes:

[0017] The acquired audio signal is subjected to phase inversion processing to obtain the first audio signal.

[0018] In some embodiments, the acquired audio signal includes a main audio signal from a main audio acquisition device and a secondary audio signal from a secondary audio acquisition device, and the generation of a first audio signal based on the acquired audio signal from the audio acquisition device includes:

[0019] The first audio signal is generated by performing phase inversion processing on either the main audio signal or the secondary audio signal, and the first audio signal is used to cancel out the one of the main audio signal and the secondary audio signal that has not undergone phase inversion processing.

[0020] In some embodiments, the acquired audio signal includes a main audio signal from a main audio acquisition device and multiple secondary audio signals from multiple secondary audio acquisition devices. Generating a first audio signal based on the acquired audio signal from the audio acquisition device includes:

[0021] Determine the target sub-audio signal from the multiple sub-audio signals;

[0022] The first audio signal is generated by performing phase inversion processing on either the main audio signal or the target sub-audio signal, and the first audio signal is used to cancel out the one of the main audio signal and the target sub-audio signal that has not undergone phase inversion processing.

[0023] In some implementations, determining the target sub-audio signal from the multiple sub-audio signals includes:

[0024] Determine the signal-to-noise ratio of each of the multiple sub-audio signals;

[0025] The sub-audio signal with the highest signal-to-noise ratio is determined as the target sub-audio signal.

[0026] In some implementations, the first audio signal has the same gain as the acquired audio signal.

[0027] According to a second aspect of the present disclosure, an audio signal processing apparatus is provided, the apparatus comprising:

[0028] The detection module is configured to detect whether the audio acquisition device of the electronic device is in operation.

[0029] The first audio signal generation module is configured to generate a first audio signal based on the audio signal acquired by the audio acquisition device, wherein the first audio signal cancels out the acquired audio signal.

[0030] According to a third aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the audio signal processing method provided in the first aspect of the present disclosure.

[0031] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a memory having a computer program stored thereon; and a processor for executing the computer program in the memory to implement the steps of the audio signal processing method mentioned in the first aspect of the present disclosure.

[0032] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: by detecting that the audio acquisition device of the electronic device is in a working state, a first audio signal that can cancel out the acquired audio signal is generated based on the audio signal acquired by the audio acquisition device. Since the first audio signal can cancel out the acquired audio signal, the monitoring program obtains the canceled signal. Therefore, even if the monitoring program detects the signal, it cannot analyze valuable information from the detected signal, or the information obtained from the detected signal is damaged. This avoids the monitoring program from obtaining user information, or reduces the harm of user information leakage to a certain extent, thereby effectively reducing the risk of user information leakage.

[0033] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0034] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0035] Figure 1 This is a flowchart illustrating an audio signal processing method according to an exemplary embodiment of the present disclosure.

[0036] Figure 2 This is a flowchart illustrating another audio signal processing method according to an exemplary embodiment of the present disclosure.

[0037] Figure 3 This is a schematic diagram illustrating the cancellation of audio signals according to an exemplary embodiment of the present disclosure.

[0038] Figure 4 This is a structural block diagram of an audio signal processing apparatus according to an exemplary embodiment of the present disclosure.

[0039] Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0040] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0041] In some cases, developers install listening programs into applications. For example, developers might install listening programs to capture user voice recordings for big data analysis of user habits and to recommend related applications. Or, they might install listening programs to capture ambient sounds around the electronic device to analyze the user's location and perform tracking. However, regardless of the purpose, listening to audio in a user's environment raises concerns about information leakage.

[0042] It is evident that how to prevent applications in electronic devices from eavesdropping on the audio in the environment in which the electronic devices are located is a problem that the field of information security has been researching and exploring for a long time.

[0043] Therefore, this disclosure provides an audio signal processing method that, by detecting that the audio acquisition device of an electronic device is in operation, generates a first audio signal that can cancel out the acquired audio signal based on the audio signal acquired by the audio acquisition device. Since the first audio signal can cancel out the acquired audio signal, the monitoring program obtains the canceled signal. Therefore, even if the monitoring program detects the signal, it cannot analyze valuable information from the detected signal, or the information obtained from the detected signal is damaged. This avoids the monitoring program from obtaining user information, or reduces the harm of user information leakage to a certain extent, thereby effectively reducing the risk of user information leakage.

[0044] Figure 1This is a flowchart illustrating an audio signal processing method according to an exemplary embodiment. The method can be applied to electronic devices, including mobile phones, laptops, tablets, desktop computers, smart speakers, smart TVs, wearable devices, smart robots, vehicle terminals, and security equipment. The method includes:

[0045] S110, The audio acquisition device of the detection electronic device is in working condition.

[0046] S120, based on the audio signal acquired by the audio acquisition device, a first audio signal is generated, and the first audio signal cancels out the acquired audio signal.

[0047] The audio acquisition device of the electronic device is in operation and can acquire audio signals.

[0048] In some implementations, the audio acquisition device may be, for example, a microphone.

[0049] Using the above method, by detecting that the audio acquisition device of the electronic device is in working condition, a first audio signal that can cancel out the acquired audio signal is generated based on the audio signal acquired by the audio acquisition device. Since the first audio signal can cancel out the acquired audio signal, the monitoring program obtains the canceled signal. Therefore, even if the monitoring program detects the signal, it cannot analyze valuable information from it, or the information obtained from analyzing the detected signal is damaged. This avoids the monitoring program obtaining user information or reduces the harm of user information leakage to a certain extent, thus effectively reducing the risk of user information leakage.

[0050] Figure 2 This is a flowchart illustrating an audio signal processing method according to an exemplary embodiment. The method can be applied to electronic devices, including mobile phones, laptops, tablets, desktop computers, smart speakers, smart TVs, wearable devices, smart robots, vehicle terminals, and security equipment, etc. The method includes:

[0051] S210, detects whether a first running voice program exists.

[0052] It is understandable that electronic devices can be used for voice calls, voice recording, and other tasks. Therefore, in some scenarios, users expect certain voice programs to call audio acquisition devices to collect audio signals. In this embodiment of the present disclosure, when these user-expected voice programs are running, they may not execute the audio signal processing method of this embodiment of the present disclosure on the audio signal, but instead execute the normal audio signal processing method to achieve normal calls or normal recordings.

[0053] For example, in the case of a call program or recording program in an electronic device, the user expects the audio acquisition device to be invoked, so that the audio signal processing method of this disclosure embodiment is not executed in the call program or recording program. The call program can be a built-in call program of the electronic device, such as a dial-up call program in a mobile phone, or a call program in a third-party application, such as the WeChat voice program.

[0054] Here, the first running voice program refers to a voice program that is not the one the user expects to run. There are several ways to detect the existence of a first running voice program.

[0055] In some implementations, the desired voice program can be determined by configuring a whitelist. In this case, it can be detected that none of the whitelisted applications are running on the electronic device. If none of the whitelisted applications are running and the audio acquisition device of the electronic device is active, then a first running voice program can be detected.

[0056] For example, when none of the voice programs expected by the user are running, such as when the user neither makes a phone call, nor records a conversation, nor uses a third-party voice function, but the audio acquisition device of the electronic device is in operation, the presence of a first running voice program can be detected.

[0057] Optionally, the whitelist configuration process can be performed by the user on the electronic device. Alternatively, the whitelist configuration process can also involve the electronic device communicating with a server to obtain the whitelist from the server and update the whitelist on the electronic device. The whitelist on the server can also be updated in real time.

[0058] In this embodiment of the disclosure, considering that there are a large number of third-party applications installed on the electronic device and that it is not easy to determine whether there is a first running voice program, the existence of a first running voice program in the electronic device can be determined by the fact that none of the voice programs in the whitelist in the electronic device are running and the audio acquisition device of the electronic device is in working condition. This can avoid configuring other voice programs besides the voice program expected by the user and simplify the configuration process of the audio processing method.

[0059] In other implementations, permissions for a specific application can be directly configured to determine whether it is the user's desired voice application. For example, a user can access an application's permission configuration interface and configure the permission indicating whether an application is the user's desired voice application to a grayed-out or green state, or, for example, to a closed or open state. This allows the user to determine whether the application is the user's desired voice application. For instance, when configured to a grayed-out or closed state, it is not the user's desired voice application; when configured to a green state or open state, it is the user's desired voice application. In this case, by detecting whether an undesirable voice application is active, the user can then determine whether a first-running voice application exists.

[0060] Furthermore, it should be noted that the program in this disclosure embodiment can refer to an APP, or it can refer to a part of the functional process of an APP. For example, in some cases, WeChat can be regarded as a program, while in other cases, the voice call function in WeChat can be regarded as a program, and the text chat function can be regarded as another program.

[0061] S220, in the presence of a first running voice program, the audio acquisition device of the detection electronic device is in working condition.

[0062] Based on the foregoing, if a first voice program is running, the next step can be to detect that the audio acquisition device of the electronic device is in a working state. Detecting that the audio acquisition device of the electronic device is in a working state can also be understood as determining that the audio acquisition device of the electronic device is in a working state.

[0063] S230, based on the audio signal acquired by the audio acquisition device, a first audio signal is generated, and the first audio signal cancels out the acquired audio signal.

[0064] In this embodiment, the acquired audio signal refers to the audio signal from the audio acquisition device. Since digital signals are more convenient for subsequent processing, the acquired audio signal can be a digital signal. Therefore, in this embodiment, if the audio acquisition device acquires an analog signal, the electronic device can first perform analog-to-digital conversion on the analog signal to obtain a digital signal, and then perform subsequent processing on the digital signal. Conversely, if the audio acquisition device acquires a digital signal, subsequent processing can be performed directly on the digital signal.

[0065] In this embodiment of the disclosure, by detecting that the audio acquisition device of the electronic device is in working state, the acquired audio signal can be obtained by the audio acquisition device, and a first audio signal can be generated based on the acquired audio signal.

[0066] The number of audio acquisition devices included in an electronic device can vary. For example, there may be one audio acquisition device, two audio acquisition devices, or more than two audio acquisition devices. Consequently, the acquisition of audio signals can also vary. For instance, the acquired audio signal may be a single audio signal from a single audio acquisition device; it may also be a main audio signal acquired by a main audio acquisition device and secondary audio signals acquired by a secondary audio acquisition device; or it may be a main audio signal acquired by a main audio acquisition device and multiple secondary audio signals acquired by multiple secondary audio acquisition devices.

[0067] In addition, it should be noted that, considering that some audio acquisition devices may not work in certain scenarios, the number of audio acquisition devices may not be exactly the same as the number of audio signals acquired.

[0068] For example, when a certain type of headphones are plugged in, the main audio acquisition device in the electronic device may be working while other audio acquisition devices are not working. In this case, even if there are other audio acquisition devices besides the main audio acquisition device, the main audio acquisition device will still be working, and the acquired audio signal will be the audio signal acquired by the main audio acquisition device.

[0069] For example, if some audio acquisition devices malfunction, only audio signals from those devices can be obtained.

[0070] Therefore, in this embodiment of the disclosure, depending on the circumstances under which the audio acquisition device acquires the audio signal, the method for generating the first audio signal based on the acquired audio signal also varies. See the detailed description below for further details.

[0071] In some implementations, the sampling rate of the audio signal acquired from the audio acquisition device can be set to 38kHz, 44kHz, 48kHz, etc.

[0072] In some implementations, the acquired audio signal includes an audio signal from an audio acquisition device. In this case, step S230 above may include the following steps:

[0073] The acquired audio signal is phase-reversed to obtain the first audio signal.

[0074] In this embodiment of the disclosure, considering that the acquired audio signal only includes one audio signal, after acquiring the acquired audio signal, the acquired audio signal can be copied to obtain another identical acquired audio signal. Then, the phase of one of the acquired audio signals is reversed to obtain the first audio signal. Then, the first audio signal can be canceled out with the acquired audio signal.

[0075] In this embodiment of the disclosure, since the acquired audio signal and the first audio signal are basically only opposite in phase and have the same other parameters, they can cancel each other out, and the cancellation effect is good. In an ideal situation, the canceled signal cannot produce sound after analysis.

[0076] In other embodiments, the acquired audio signal may include a main audio signal from the main audio acquisition device and a secondary audio signal from the secondary audio acquisition device. In this case, step S230 above may include the following steps:

[0077] Either the main audio signal or the secondary audio signal is phase-inverted to generate a first audio signal, which is used to cancel out the other of the main audio signal and the secondary audio signal that has not been phase-inverted.

[0078] In this embodiment of the disclosure, when the acquired audio signal includes two audio signals, either of the two audio signals can be directly phase-inverted. The audio signal that has undergone phase inversion becomes the first audio signal after phase inversion. Thus, the first audio signal can be canceled out from the main audio signal and the audio signals in the secondary audio signal that have not undergone phase inversion.

[0079] For example, suppose the main audio signal is phase-inverted to obtain the first audio signal, then the first audio signal and the secondary audio signal can be canceled out.

[0080] For example, suppose the secondary audio signal is phase-inverted to obtain the first audio signal. Then the first audio signal and the primary audio signal can be canceled out.

[0081] In other embodiments, the acquired audio signals include a main audio signal from a main audio acquisition device and multiple secondary audio signals from multiple secondary audio acquisition devices. In this case, step S230 above may include the following steps:

[0082] The target sub-audio signal is determined from the multiple sub-audio signals. The phase inversion process is performed on either the main audio signal or the target sub-audio signal to generate a first audio signal. The first audio signal is used to cancel out the one of the main audio signal and the target sub-audio signal that has not undergone phase inversion process.

[0083] In this embodiment of the disclosure, since only two audio signals are ultimately needed, when there is a main audio signal and multiple secondary audio signals, the main audio signal and one target secondary audio signal can be retained. Then, the signal is processed in a similar manner to the aforementioned main audio signal and one secondary audio signal, that is, the phase of either the main audio signal or the target secondary audio signal is reversed to obtain the first audio signal. Then, the audio signals of the main audio signal and the target secondary audio signal that have not undergone phase reversal can be canceled out with the first audio signal.

[0084] There are several ways to determine the target sub-audio signal from multiple sub-audio signals.

[0085] Optionally, any one of the secondary audio signals can be selected as the target secondary audio signal.

[0086] Optionally, determining the target sub-audio signal from multiple sub-audio signals may include the steps of: determining the signal-to-noise ratio (SNR) of each of the multiple sub-audio signals, and determining the sub-audio signal with the highest SNR as the target sub-audio signal.

[0087] In this embodiment, the secondary audio signal with a higher signal-to-noise ratio has better sound quality and is closer to the main audio signal. Determining the secondary audio signal with the highest signal-to-noise ratio as the target secondary audio signal can improve the cancellation effect of subsequent cancellation and enhance the effect of preventing information leakage.

[0088] In cases where the electronic device includes two or more audio acquisition devices, the following scenarios may exist.

[0089] When users run desired voice programs on electronic devices, they are typically making phone calls or recording audio. In these scenarios, there is no need to cancel out the audio signal. Furthermore, due to the structural design of electronic devices, the mouth is usually relatively close to the main audio acquisition device and relatively far from the secondary audio acquisition device, resulting in a significant difference between the audio signals acquired by the main and secondary audio acquisition devices.

[0090] When a user is not using the electronic device to run the desired voice program—meaning none of the desired voice programs are running—usually not in a scenario like making a call or recording, audio signal cancellation is necessary. In this case, due to the structural design of electronic devices, the mouth is typically far from both the main and secondary audio acquisition devices, resulting in a relatively small difference in their relative distances. This minimizes the difference between the audio signals acquired by the main and secondary audio acquisition devices. Based on this characteristic, the audio signals acquired by the audio acquisition devices can be effectively canceled, preventing the leakage of user information.

[0091] Furthermore, in some special cases, considering that when the user is not using an electronic device to run the desired voice program, there may be a situation where the mouth is relatively close to the main audio acquisition device and relatively far from the secondary audio acquisition device, resulting in a large difference between the audio signals acquired by the main audio acquisition device and the audio signals acquired by the secondary audio acquisition device. In this case, the main difference between the main audio signal and the secondary audio signal lies in the difference in gain. Therefore, in order to improve the mutual cancellation effect, in some embodiments, the gain of the first audio signal and the acquired audio signal can be made the same.

[0092] In this embodiment of the disclosure, after making the gain of the first audio signal and the acquired audio signal the same, the difference between the first audio signal and the acquired audio signal is small. After canceling the first audio signal and the acquired audio signal, the audio signal acquired by the audio acquisition device can also be effectively canceled, thus avoiding the leakage of user information.

[0093] Gain can be loudness.

[0094] Therefore, in some implementations, the gain of the first audio signal can be made the same as that of the acquired audio signal by configuring a loudness parameter. For example, the loudness parameter can be a loudness increase amount or a loudness increase percentage.

[0095] S240, store the first audio signal and the signal after canceling the acquired audio signal, and the stored signal is used to be acquired by the first running voice program.

[0096] In some implementations, the application obtains the audio signal by calling a relevant interface to retrieve the audio signal from a preset memory location. That is, after obtaining the audio signal, it can send the audio signal to the preset memory location for caching. Therefore, in this embodiment of the present disclosure, after canceling the first audio signal and the acquired audio signal, the canceled signal can be stored so that it can be obtained by the first running voice program.

[0097] In some implementations, the first audio signal and the signal that cancels out the acquired audio signal can be stored in different memory locations so that they can be acquired by the first running voice program and the user-expected voice program, respectively.

[0098] As can be seen, in this embodiment of the present disclosure, since the signal acquired by the first running voice program is a canceled signal, even if the first running voice program acquires it, it cannot analyze it to obtain valuable information, or the information analyzed from the target audio signal is damaged. This avoids the first running voice program acquiring user information, or reduces the harm of user information leakage to a certain extent. In some embodiments, the cancellation of the first audio signal and the acquired audio signal can be achieved by superimposing their waveforms.

[0099] Please see Figure 3 This illustrates a schematic diagram of canceling out the first audio signal with the acquired audio signal, such as... Figure 3 As shown, the first audio signal is canceled out with the acquired audio signals (assuming they are a1 and a2 respectively) to obtain the canceled signal a3. When a1 and a2 are completely canceled out, the canceled signal a3 cannot produce sound after being analyzed.

[0100] Furthermore, considering that the first audio signal and the acquired audio signal, which cancel each other out, may not have completely identical parameters other than phase reversal, for example, in the case of an electronic device including two or more audio acquisition devices, there may be slight differences between the audio signals acquired by the main audio acquisition device and each of the auxiliary audio acquisition devices. Also, nonlinear processing during signal processing may lead to differences in other signal parameters. In such cases, directly canceling the first audio signal and the acquired audio signal may still leave some sound. Therefore, to further improve the information leakage prevention effect, in some embodiments, after canceling the first audio signal and the acquired audio signal, the canceled signal can be further subjected to noise reduction processing, such as filtering, to filter out the audio signal and obtain a target audio signal that cannot produce sound. This target audio signal is used to be acquired by the first running voice program.

[0101] Using the above method, after canceling the first audio signal with the acquired audio signal, the canceled audio signal is further filtered. Under ideal conditions, the sound can be almost completely eliminated, thereby further improving the effect of preventing information leakage and effectively reducing the risk of user information leakage.

[0102] Furthermore, based on the same considerations mentioned above, in order to further improve the effectiveness of preventing information leakage and effectively reduce the risk of user information leakage, in some other embodiments, the method of this disclosure may further include the following steps:

[0103] Based on a preset signal generation strategy, a second audio signal is generated. The second audio signal is used to superimpose the signal after canceling out the first audio signal and the acquired audio signal. The second audio signal is different from the acquired audio signal.

[0104] Optionally, the preset signal generation strategy can be a random signal generation strategy, that is, randomly generating a second audio signal, and then superimposing the randomly generated second audio signal with the signal obtained by canceling out the first audio signal and the acquired audio signal. Since the second audio signal is different from the acquired audio signal, it can interfere with the signal obtained by the first running voice program, so that the first running voice program cannot correctly parse the user's original voice information, thereby further improving the effect of preventing information leakage and effectively reducing the risk of user information leakage.

[0105] Optionally, the preset signal generation strategy can also be to perform linear or nonlinear transformations, weighting, and filtering on the acquired audio signal to obtain an interfering second audio signal. The interfering second audio signal is then superimposed with the signal obtained after canceling the first audio signal and the acquired audio signal. This can also prevent the first running voice program from correctly parsing the user's original voice information after acquiring the signal, thereby further improving the effect of preventing information leakage and effectively reducing the risk of user information leakage.

[0106] In some implementations, hardware circuit modules can be configured to perform some steps of the audio signal processing method in the embodiments of this disclosure. For example, hardware circuit modules can be configured to perform the step of acquiring audio signals based on an audio acquisition device and generating a first audio signal.

[0107] In some implementations, the hardware circuit module can be modularized into a chip for easy mass production.

[0108] Figure 4 This is a structural block diagram of an audio signal processing apparatus 300 according to an exemplary embodiment. (Refer to...) Figure 4 The device includes: a detection module 310 and a first audio signal generation module 320, wherein:

[0109] The detection module 310 is configured to detect that the audio acquisition device of the electronic device is in operation.

[0110] The first audio signal generation module 320 is configured to generate a first audio signal based on the audio signal acquired by the audio acquisition device, wherein the first audio signal cancels out the acquired audio signal.

[0111] In some implementations, the detection module 310 is further configured to detect the presence of a first running voice program; if the first running voice program is present, it detects that the audio acquisition device of the electronic device is in operation.

[0112] In some embodiments, the device 300 further includes:

[0113] The storage module is configured to store the first audio signal and the signal obtained by canceling the acquired audio signal, and the stored signal is used to be acquired by the first running voice program.

[0114] In some embodiments, the device 300 further includes:

[0115] The second audio signal generation module is configured to generate a second audio signal based on a preset signal generation strategy. The second audio signal is used to be superimposed on the signal after the first audio signal and the acquired audio signal are canceled out. The second audio signal is different from the acquired audio signal.

[0116] In some embodiments, the device 300 further includes:

[0117] The filtering module is configured to filter the first audio signal and the signal obtained after canceling the acquired audio signal to obtain the target audio signal.

[0118] In some embodiments, the acquired audio signal includes an audio signal from an audio acquisition device, and the first audio signal generation module 320 includes:

[0119] The first generation submodule is configured to perform phase inversion processing on the acquired audio signal to obtain the first audio signal.

[0120] In some embodiments, the acquired audio signal includes a main audio signal from the main audio acquisition device and a secondary audio signal from the secondary audio acquisition device, and the first audio signal generation module 320 includes:

[0121] The second generation submodule is configured to perform phase inversion processing on either the main audio signal or the secondary audio signal to generate the first audio signal, which is used to cancel out the one of the main audio signal and the secondary audio signal that has not undergone phase inversion processing.

[0122] In some embodiments, the acquired audio signal includes a main audio signal from a main audio acquisition device and multiple secondary audio signals from multiple secondary audio acquisition devices, and the first audio signal generation module 320 includes:

[0123] The target sub-audio signal determination submodule is configured to determine the target sub-audio signal from the multiple sub-audio signals;

[0124] The third generation submodule is configured to perform phase inversion processing on either the main audio signal or the target sub-audio signal to generate the first audio signal, which is used to cancel out the one of the main audio signal and the target sub-audio signal that has not undergone phase inversion processing.

[0125] In some implementations, the target sub-audio signal determination submodule is further configured to determine the signal-to-noise ratio (SNR) of each of the multiple sub-audio signals; and to determine the sub-audio signal with the highest SNR as the target sub-audio signal.

[0126] In some implementations, the first audio signal has the same gain as the acquired audio signal.

[0127] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0128] The apparatus employing this embodiment detects that the audio acquisition device of the electronic device is in operation, and generates a first audio signal that can cancel out the acquired audio signal based on the audio signal acquired by the audio acquisition device. Since the first audio signal can cancel out the acquired audio signal, the monitoring program obtains the canceled signal. Therefore, even if the monitoring program detects the signal, it cannot analyze valuable information from the detected signal, or the information obtained from the detected signal is damaged. This avoids the monitoring program from obtaining user information, or reduces the harm of user information leakage to a certain extent, thereby effectively reducing the risk of user information leakage.

[0129] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the audio signal processing method provided in this disclosure.

[0130] Figure 5 This is a block diagram illustrating an electronic device 400 according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, laptop, tablet computer, desktop computer, smart speaker, smart TV, wearable device, smart robot, vehicle terminal, and security equipment, etc.

[0131] Reference Figure 5 The electronic device 400 may include one or more of the following components: processing component 402, memory 404, power component 406, multimedia component 408, audio component 410, input / output (I / O) interface 412, sensor component 414, and communication component 416.

[0132] Processing component 402 typically controls the overall operation of electronic device 400, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the audio signal processing method described above. Furthermore, processing component 402 may include one or more modules to facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.

[0133] Memory 404 is configured to store various types of data to support the operation of electronic device 400. Examples of this data include instructions for any application or method operating on electronic device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0134] Power component 406 provides power to various components of electronic device 400. Power component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 400.

[0135] Multimedia component 408 includes a screen that provides an output interface between electronic device 400 and a user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera and / or a rear-facing camera. When electronic device 400 is in an operating mode, such as a shooting mode or video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0136] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when electronic device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

[0137] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0138] Sensor assembly 414 includes one or more sensors for providing state assessments of various aspects of electronic device 400. For example, sensor assembly 414 may detect the on / off state of electronic device 400, the relative positioning of components such as the display and keypad of electronic device 400, changes in position of electronic device 400 or a component of electronic device 400, the presence or absence of user contact with electronic device 400, orientation or acceleration / deceleration of electronic device 400, and temperature changes of electronic device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0139] Communication component 416 is configured to facilitate wired or wireless communication between electronic device 400 and other devices. Electronic device 400 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0140] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the audio signal processing method described above.

[0141] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by a processor 420 of an electronic device 400 to complete the audio signal processing method described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0142] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described audio signal processing method when executed by the programmable device.

[0143] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0144] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An audio signal processing method, characterized in that, include: Detect whether there is a first running voice program, which is a voice program that is not in a running state other than the voice program expected by the user; In the presence of the first running voice program, the audio acquisition device of the detection electronic device is in working condition; Based on the audio signal acquired by the audio acquisition device, a first audio signal is generated, and the first audio signal cancels out the acquired audio signal; The first audio signal and the signal obtained by canceling the acquired audio signal are stored, and the stored signal is used to be acquired by the first running voice program; Based on a preset signal generation strategy, a second audio signal is generated. The second audio signal is used to superimpose the signal obtained by canceling out the first audio signal and the acquired audio signal. The second audio signal is different from the acquired audio signal.

2. The method according to claim 1, characterized in that, The method further includes: The first audio signal and the acquired audio signal after cancellation are filtered to obtain the target audio signal.

3. The method according to claim 1 or 2, characterized in that, The acquired audio signal includes an audio signal from an audio acquisition device, and the generation of a first audio signal based on the acquired audio signal from the audio acquisition device includes: The acquired audio signal is subjected to phase inversion processing to obtain the first audio signal.

4. The method according to claim 1 or 2, characterized in that, The acquired audio signals include a main audio signal from the main audio acquisition device and a secondary audio signal from the secondary audio acquisition device. Generating a first audio signal based on the acquired audio signals from the audio acquisition device includes: The first audio signal is generated by performing phase inversion processing on either the main audio signal or the secondary audio signal, and the first audio signal is used to cancel out the one of the main audio signal and the secondary audio signal that has not undergone phase inversion processing.

5. The method according to claim 1 or 2, characterized in that, The acquired audio signals include a main audio signal from a main audio acquisition device and multiple secondary audio signals from multiple secondary audio acquisition devices. The generation of a first audio signal based on the acquired audio signals from the audio acquisition devices includes: Determine the target sub-audio signal from the multiple sub-audio signals; The first audio signal is generated by performing phase inversion processing on either the main audio signal or the target sub-audio signal, and the first audio signal is used to cancel out the one of the main audio signal and the target sub-audio signal that has not undergone phase inversion processing.

6. The method according to claim 5, characterized in that, Determining the target sub-audio signal from the multiple sub-audio signals includes: Determine the signal-to-noise ratio of each of the multiple sub-audio signals; The sub-audio signal with the highest signal-to-noise ratio is determined as the target sub-audio signal.

7. The method according to claim 1 or 2, characterized in that, The first audio signal has the same gain as the acquired audio signal.

8. An audio signal processing device, characterized in that, The device includes: The detection module is configured to detect whether a first running voice program exists, wherein the first running voice program is a voice program that is not in a running state other than the voice program expected by the user; if the first running voice program exists, the audio acquisition device of the electronic device is detected to be in a working state. The first audio signal generation module is configured to generate a first audio signal based on the audio signal acquired by the audio acquisition device, wherein the first audio signal cancels out the acquired audio signal; The storage module is configured to store the first audio signal and the signal after the acquired audio signal is canceled out, and the stored signal is used to be acquired by the first running voice program; The second audio signal generation module is configured to generate a second audio signal based on a preset signal generation strategy. The second audio signal is used to be superimposed on the signal after the first audio signal and the acquired audio signal are canceled out. The second audio signal is different from the acquired audio signal.

9. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-7.

Images