Method, apparatus and electronic device for processing voice service

Abstract

The application provides a method, device and electronic equipment for processing voice service, and relates to the technical field of terminals, which comprises the following steps: collecting a first audio signal for voice service through a microphone; outputting a second audio signal based on the first audio signal, wherein the signal-to-noise ratio of the second audio signal is greater than that of the first audio signal if the first audio signal comprises a laser attack signal; and processing the voice service based on the second audio signal. The technical scheme provided by the application can reduce the risk of laser attack, improve the security of electronic equipment and user data, and ensure the convenience of the electronic equipment without additional reminders or confirmations for the user, thereby improving the user experience and avoiding economic losses of the company brand.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method, apparatus and electronic device for processing voice services. Background Technology

[0002] Electronic devices are typically equipped with microphones. These microphones capture user speech, producing audio signals, which are then used to process voice data. However, due to the limitations of microphone hardware, electronic devices may capture both genuine user speech and laser attack signals in response to lasers. These laser attack signals could interfere with the normal operation of the electronic device, create vulnerabilities, and threaten user privacy and security.

[0003] In some technologies, electronic devices can add additional prompts or confirmation steps for users to verify the reliability of the collected audio signals. However, this sacrifices the convenience of the electronic device, and the cumbersome operation and false alarms can cause users to question the reliability of the electronic device and the company brand, ultimately leading to refusal to use or disabling the relevant functions of the electronic device, thus causing economic losses to the company brand. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, and electronic device for processing voice services, which can reduce the risk of laser attacks, improve the security of electronic devices and user data, and eliminate the need for additional reminders or confirmations to users, ensuring the convenience of electronic devices, enhancing user experience, and avoiding economic losses to the company's brand.

[0005] To achieve the above objectives, in a first aspect, embodiments of this application provide a method for processing voice services, the method comprising:

[0006] The first audio signal for voice services is acquired via a microphone;

[0007] A second audio signal is output based on the first audio signal, wherein if the first audio signal includes a laser attack signal, the signal-to-noise ratio of the second audio signal is greater than that of the first audio signal;

[0008] The voice service is processed based on the second audio signal.

[0009] The first audio signal may include audio signals with frequencies lower than the preset upper limit of the pickup frequency (such as the actual voice of the user), i.e., normal audio signals, or abnormal audio signals such as laser attack signals generated by laser attacks and / or ultrasonic attack signals generated by ultrasonic attacks.

[0010] It should be noted that the signal-to-noise ratio (SNR) refers to the ratio of signal to noise that an electronic system needs to process. In the embodiments of this application, the higher the SNR in an audio signal, the better the signal quality of the audio signal, and the higher the accuracy of the voice information carried by the audio signal in the subsequent voice service.

[0011] In this embodiment, when a first audio signal is acquired via a microphone, a second audio signal can be output based on the first audio signal. This results in a higher signal-to-noise ratio (SNR) for the second audio signal compared to the first audio signal, even if the first audio signal includes a laser attack signal. Therefore, when processing voice services based on the second audio signal, the laser attack signal in the second audio signal is more difficult to identify, thus reducing the risk of laser attacks and improving the security of electronic devices and user data. Since no additional reminders or confirmations are required for the user, the convenience of the electronic device is not reduced, the user experience is improved, and economic losses to the company's brand are avoided.

[0012] In some embodiments, the electronic device can determine whether the acquisition of the first audio signal is interfered with (or determine whether the first audio signal includes laser attack signals generated by laser attacks and / or ultrasonic attack signals generated by ultrasonic attacks). If so, it outputs a second audio signal based on the first signal and processes voice services based on the second audio signal; otherwise, it processes voice services based on the first audio signal and no longer outputs the second audio signal, thereby improving the accuracy of noise addition and reducing interference with the normal interaction between the electronic device and the user. In some embodiments, the electronic device can determine whether the interference to the acquisition of the first audio signal is a laser attack or an ultrasonic attack (or determine whether the first audio signal specifically includes a laser attack signal or an ultrasonic attack signal).

[0013] In some embodiments, the electronic device can acquire the root mean square (RMS) value of the amplitude of the first audio signal. If the RMS amplitude is less than a preset first RMS threshold, it is determined that the interference affecting the acquisition of the first audio signal is a laser attack; otherwise, it is determined that the interference affecting the acquisition of the first audio signal is specifically an ultrasonic attack. In some embodiments, the first RMS threshold can be -50 dB. Of course, in practical applications, the first RMS threshold can also be other values, and this application embodiment does not limit the value of the first RMS threshold.

[0014] In some embodiments, the electronic device can divide the first audio signal into multiple frames according to time sequence (e.g., each 100ms segment of the first audio signal is considered one frame), determine the RMS amplitude of each frame, and then determine a first difference range of the RMS amplitudes of the multiple frames. If the first difference range is less than a preset second RMS threshold, it is determined that the interference received by the operation of acquiring the first audio signal is a laser attack; otherwise, it is determined that the interference received by the operation of acquiring the first audio signal is an ultrasonic attack. In some embodiments, the second RMS threshold can be 3dB. Of course, in practical applications, the second RMS threshold can also be other values, and this application embodiment does not limit the value of the second RMS threshold.

[0015] In some embodiments, the step of outputting a second audio signal based on the first audio signal includes:

[0016] Acquire a third audio signal, wherein the third audio signal is a noise signal with a frequency range including at least a portion of frequencies from 0Hz to 16KHz;

[0017] Based on the third audio signal, the first audio signal is subjected to noise addition processing to obtain the second audio signal.

[0018] The electronic device can acquire the stored third audio signal. Alternatively, the electronic device can generate the third audio signal using hardware and / or software for generating noise signals.

[0019] In some embodiments, the third audio signal may be a white noise signal.

[0020] In some embodiments, the frequency range of the third audio signal can be 0Hz-8KHz, 0Hz-16KHz, 300Hz-8KHz, or 300Hz-16KHz.

[0021] In some embodiments, the noise addition processing of the first audio signal based on the third audio signal includes:

[0022] The sound corresponding to the third audio signal is played through a speaker, and the second audio signal is acquired through a microphone, wherein the duration of playing the sound corresponding to the third audio signal at least partially overlaps with the duration of acquiring the second audio signal.

[0023] Since sound waves in nature belong to the analog domain, when an electronic device plays sound based on a third audio signal, and the duration of playing the sound at least partially overlaps with the duration of the collected sound, the second audio signal can be obtained by the microphone. That is, the first audio signal is noise-added in the analog domain by the third audio signal, so that the second audio signal includes one or more of the normal audio signal, laser attack signal, and ultrasonic attack signal, as well as the noise signal.

[0024] In some embodiments, the noise addition processing of the first audio signal based on the third audio signal includes:

[0025] The third audio signal is modulated onto the fourth audio signal to obtain the fifth audio signal, wherein the frequencies of the fourth audio signal and the fifth audio signal are greater than 16KHz;

[0026] The sound corresponding to the fifth audio signal is played through a speaker, and the second audio signal is acquired through the microphone, wherein the duration of playing the sound corresponding to the fifth audio signal at least partially overlaps with the duration of acquiring the second audio signal.

[0027] Since the frequencies of the fourth and fifth audio signals are greater than 16kHz, the played sound is less likely to be perceived by the user, reducing the disturbance to the user caused by the noise addition process.

[0028] The fourth audio signal can be a carrier signal, used as a carrier for another signal to be transmitted; the third audio signal can be a modulated signal, i.e., another signal to be transmitted; and the fifth audio signal is a modulated signal. In some embodiments, the frequency range of the fourth and fifth audio signals may include 24 kHz and / or 36 kHz. Of course, in practical applications, the frequency range of the fourth and fifth audio signals may include other frequencies.

[0029] In some embodiments, the method further includes:

[0030] The sixth audio signal and the second audio signal acquired by the microphone are superimposed in the digital domain to obtain a new second audio signal. The sixth audio signal is a noise signal with a frequency range including at least some frequencies in the range of 0Hz-16KHz.

[0031] The process of processing the voice service based on the second audio signal includes:

[0032] The voice service is processed based on the new second audio signal.

[0033] The third audio signal and the sixth audio signal can be the same audio signal or different audio signals. That is, when adding noise in both the digital and analog domains at the same time, the audio signals used for adding noise in the digital domain and the analog domain can be the same audio signal or different audio signals.

[0034] In some embodiments, the noise addition processing of the first audio signal based on the third audio signal includes:

[0035] The third audio signal and the first audio signal are superimposed in the digital domain to obtain the second audio signal.

[0036] In this method, by superimposing the first and third audio signals in the digital domain—that is, by interfering with the first audio signal through the third audio signal in the digital domain—the generated second audio signal includes noise signals in addition to one or more of the following: normal audio signals, laser attack signals, and ultrasonic attack signals. Therefore, the signal-to-noise ratio (SNR) of the second audio signal is greater than that of the first audio signal. Since the normal audio signal is obtained by the microphone from sounds with frequencies below the upper limit of the pickup frequency, while the abnormal audio signal is obtained by demodulating ultrasonic waves and / or lasers, the normal audio signal is more robust than the abnormal audio signal. Therefore, the noise signal interferes less with the normal audio signal but significantly interferes with the laser or ultrasonic attack signals, thereby reducing the risk of electronic devices being attacked by lasers or ultrasonic waves and improving the security of electronic devices and user data.

[0037] In some embodiments, the noise addition processing of the first audio signal based on the third audio signal includes:

[0038] If it is determined that the microphone is under laser attack, then the first audio signal is noise-added based on the third audio signal.

[0039] In some embodiments, the method further includes:

[0040] If it is determined that the microphone is under ultrasonic attack, then the first audio signal is subjected to noise enhancement based on the seventh audio signal, wherein the power of the seventh audio signal is greater than the power of the third audio signal.

[0041] Since a high laser power could burn out a microphone when it is attacked by a laser, the laser power used in the laser attack is relatively low. Therefore, in another embodiment, the electronic device can add noise to the first audio signal based on a third audio signal when it is determined that the microphone is under laser attack; and add noise to the first audio signal based on a seventh audio signal when it is determined that the microphone is under ultrasonic attack.

[0042] In some embodiments, the power of the third audio signal is positively correlated with the probability of being subjected to the laser or ultrasonic attack, which means that the greater the probability of being subjected to the laser or ultrasonic attack, the greater the intensity of the added noise, i.e., the greater the signal-to-noise ratio of the second audio signal is compared to that of the first audio signal.

[0043] Secondly, embodiments of this application provide an apparatus for processing voice services, the apparatus including an audio module and a voice service module;

[0044] The audio module is used to acquire a first audio signal for voice services through a microphone and output a second audio signal based on the first audio signal. If the first audio signal includes a laser attack signal, the signal-to-noise ratio of the second audio signal is greater than that of the first audio signal.

[0045] The voice service module is used to process the voice service based on the second audio signal.

[0046] In some embodiments, the audio module is specifically used for:

[0047] Acquire a third audio signal, wherein the third audio signal is a noise signal with a frequency range including at least a portion of frequencies from 0Hz to 16kHz;

[0048] Based on the third audio signal, the first audio signal is subjected to noise addition processing to obtain the second audio signal.

[0049] In some embodiments, the audio module is specifically used for:

[0050] The sound corresponding to the third audio signal is played through a speaker, and the second audio signal is acquired through a microphone, wherein the duration of playing the sound corresponding to the third audio signal at least partially overlaps with the duration of acquiring the second audio signal.

[0051] In some embodiments, the audio module is specifically used for:

[0052] The third audio signal is modulated onto the fourth audio signal to obtain the fifth audio signal, wherein the frequencies of the fourth audio signal and the fifth audio signal are greater than 16KHz;

[0053] The sound corresponding to the fifth audio signal is played through a speaker, and the second audio signal is acquired through the microphone, wherein the duration of playing the sound corresponding to the fifth audio signal at least partially overlaps with the duration of acquiring the second audio signal.

[0054] In some embodiments, the audio module is specifically used for:

[0055] The sixth audio signal and the second audio signal acquired by the microphone are superimposed in the digital domain to obtain a new second audio signal. The sixth audio signal is a noise signal with a frequency range including at least some frequencies in the range of 0Hz-16KHz.

[0056] The voice service module is specifically used for:

[0057] The voice service is processed based on the new second audio signal.

[0058] In some embodiments, the audio module is specifically used for:

[0059] The third audio signal and the first audio signal are superimposed in the digital domain to obtain the second audio signal.

[0060] In some embodiments, the apparatus further includes a processing module;

[0061] If the processing module determines that the microphone is under laser attack, it triggers the audio module to add noise to the first audio signal based on the third audio signal.

[0062] In some embodiments, if the processing module determines that the microphone is under ultrasonic attack, it triggers the audio module to add noise to the first audio signal based on a seventh audio signal, wherein the power of the seventh audio signal is greater than the power of the third audio signal.

[0063] In some embodiments, the power of the third audio signal is positively correlated with the credibility of being subjected to the laser attack or ultrasonic attack.

[0064] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method described in any one of the first aspects when the computer program is invoked.

[0065] Fourthly, embodiments of this application provide a chip system including a processor coupled to a memory, wherein the processor executes a computer program stored in the memory to implement the method described in any one of the first aspects above.

[0066] The chip system can be a single chip or a chip module composed of multiple chips.

[0067] Fifthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any one of the first aspects above.

[0068] In a sixth aspect, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the method described in any one of the first aspects.

[0069] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0070] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0071] Figure 2 This is a schematic diagram illustrating the principle of microphone pickup provided in an embodiment of this application;

[0072] Figure 3 A schematic diagram illustrating the principle of a laser attack microphone provided in an embodiment of this application;

[0073] Figure 4 This is a frequency diagram of an audio signal provided in an embodiment of this application;

[0074] Figure 5 This is a frequency diagram of another audio signal provided in an embodiment of this application;

[0075] Figure 6 This is a frequency diagram of another audio signal provided in an embodiment of this application;

[0076] Figure 7 A flowchart illustrating a method for processing voice services provided in an embodiment of this application;

[0077] Figure 8 A flowchart illustrating another method for processing voice services provided in an embodiment of this application;

[0078] Figure 9 A flowchart illustrating another method for processing voice services provided in an embodiment of this application;

[0079] Figure 10 A flowchart illustrating another method for processing voice services provided in an embodiment of this application;

[0080] Figure 11 A flowchart illustrating another method for processing voice services provided in an embodiment of this application;

[0081] Figure 12 A flowchart illustrating another method for processing voice services provided in an embodiment of this application;

[0082] Figure 13 This is a schematic diagram of the structure of an apparatus for processing voice services provided in an embodiment of this application. Detailed Implementation

[0083] The method for processing voice services provided in this application can be applied to electronic devices equipped with microphones, such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), smart home devices, and smart locks. This application does not impose any restrictions on the specific type of electronic device.

[0084] Figure 1 This is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of this application. The electronic device 100 may include a processor 110, a memory 120, a communication module 130, an audio module 140, a speaker 150, a microphone 160, and a voice service module 170, etc.

[0085] The processor 110 may include one or more processing units, and the memory 120 is used to store program code and data. In this embodiment, the processor 110 can execute computer execution instructions stored in the memory 120 to control and manage the operation of the electronic device 100. For example, the processor 110 may include an application processor (AP) and / or a neural network processing unit (NPU), etc.

[0086] NPU stands for Neural Network (NN) Computing Processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as speech recognition and text understanding.

[0087] The communication module 130 can be used for communication between various internal modules of the electronic device 100, or for communication between the electronic device 100 and other external electronic devices. For example, the communication module 130 may include interfaces. For instance, the communication module 130 may include an inter-integrated circuit sound (I2S) interface, which can be used for audio communication. In some embodiments, the processor 110 can be coupled to the audio module 140 via an I2S bus to achieve communication between the processor 110 and the audio module 140.

[0088] Alternatively, the communication module 130 may include audio devices, radio frequency circuits, Bluetooth chips, wireless fidelity (Wi-Fi) chips, near-field communication (NFC) modules, etc., which can enable the electronic device 100 to interact with other electronic devices in a variety of different ways.

[0089] Electronic device 100 can implement audio functions through audio module 140, speaker 150, microphone 160, application processor, etc., such as music playback and recording.

[0090] Audio module 140 is used to convert digital audio signals into analog audio signals and vice versa, wherein both digital and analog audio signals are electrical signals. Audio module 140 can also be used for encoding and decoding audio signals. In some embodiments, audio module 140 may include an amplifier, a low-pass filter, an analog-to-digital converter (ADC), and a digital-to-analog converter (DAC). The amplifier can be used to amplify the gain of the analog audio signal, the low-pass filter can be used to filter out high-frequency audio signals (such as audio signals above the pickup frequency limit), the ADC can be used to convert the analog audio signal into a digital audio signal, and the DAC can be used to convert the digital audio signal into an analog audio signal. In some embodiments, audio module 140 may be located in processor 110, or some functional modules of audio module 140 may be located in processor 110.

[0091] It should be noted that the upper limit of the pickup frequency can be set in advance by the manufacturer of the electronic device 100, for example, the upper limit of the pickup frequency can be 16KHz (kilohertz). Of course, in practical applications, the upper limit of the pickup frequency can also be other values, and this application embodiment does not limit the specific value of the upper limit of the pickup frequency.

[0092] The speaker 150, also known as a "loudspeaker," is used to convert electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 150.

[0093] Microphone 160, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. In some embodiments, microphone 160 may include a micro-electro-mechanical system (MEMS) microphone. MEMS is a miniature device or system that integrates microsensors, microactuators, micromechanical structures, micropower sources, signal processing and control circuits, high-performance electronic integrated devices, interfaces, and communication functions. A MEMS microphone is a microphone based on MEMS technology. In some embodiments, the microphone may include multiple microphone units.

[0094] The voice service module 170 can be used for various voice services based on the audio signal processing of the audio module 140. Voice services can be services performed based on the user's voice. In some embodiments, voice services may include one or more of voice activity detection (VAD), keyword spotting (KWS), and automatic speech recognition (ASR). VAD is a voice processing technology that can detect whether the audio signal to be detected includes voice information from the user. In some embodiments, VAD can be used to accurately locate the start and end points of voice information from the audio signal, and can also remove silence and noise to obtain truly effective voice content. KWS can be used to detect predefined words or phrases in the audio signal, which can be used as voice commands to trigger corresponding functions. ASR is a technology that converts voice information into text information. In some embodiments, the converted text information can be used by the electronic device to determine the user's intent or instruction, and then to engage in dialogue with the user or perform corresponding operations according to the user's intent or instruction. Of course, in practical applications, the voice service module 170 can also be used to process more or fewer voice services; this application embodiment does not limit the type of voice service. In some embodiments, the voice service module 170 may be coupled to the audio module 140, or may be located in the processor 110, or some functional modules of the voice service module 170 may be located in the processor 110.

[0095] It should be understood that, in addition to Figure 1 In addition to the various components or modules listed, the embodiments of this application do not specifically limit the structure of the electronic device 100. In other embodiments of this application, the electronic device 100 may also include more or fewer components than those shown in the figures, or combine some components, or split some components, or have different component arrangements. The components shown in the figures may be implemented in hardware, software, or a combination of software and hardware.

[0096] The following example, using the scenario of waking up electronic device 100, illustrates the roles and workflows of some components included in electronic device 100.

[0097] Electronic device 100 is in sleep or standby mode. The user wishes to wake up electronic device 100 and therefore utters the voice command "Hey Celia." Electronic device 100 picks up the voice command through microphone 160, obtaining a corresponding analog audio signal. Audio module 140 performs analog-to-digital conversion on the analog audio signal to obtain a corresponding digital audio signal. Voice service module 170 first performs VAD detection on the digital audio signal to determine that it contains voice information. Then, it performs KWS recognition on the digital audio signal (or a portion of the digital audio signal containing only the voice information) to determine that it includes the preset wake-up word "Celestial." Therefore, it requests processor 110 to switch electronic device 100 to wake-up mode. Voice service module 170 can also generate another digital audio signal for user interaction, which includes the voice message "Hey Celia" used to reply to "Celestial." Audio module 140 performs digital-to-analog conversion on this other digital audio signal to obtain another corresponding analog audio signal, which is then output through speaker 150, i.e., uttering the voice command "Hey Celia," thus notifying the user that they have switched to wake-up mode.

[0098] To facilitate understanding of the technical solutions in the embodiments of this application, the application scenarios of the embodiments of this application will be introduced first below.

[0099] Please refer to Figure 2 This is a schematic diagram illustrating the principle of microphone pickup provided in an embodiment of this application. Microphone 160 ( Figure 2 The microphone (not shown) may include a vibration component 200, which in some embodiments may be a diaphragm. Since sound waves are mechanical waves, when a sound source emits sound (such as a user speaking), the sound will propagate in all directions. When the sound reaches the microphone 160, the energy of the sound will cause the vibration component 200 to vibrate mechanically, and then generate an analog electrical signal (i.e., an analog audio signal) corresponding to the mechanical vibration of the vibration component 200 through electromagnetic induction or other means.

[0100] Please refer to the following: Figure 3 When the vibrating component 200 is irradiated by a laser, the pulse energy of the laser will also compress the vibrating component 200, causing the vibrating component 200 to also vibrate mechanically. Correspondingly, the microphone 160 ( Figure 3 (Not shown in the image) It also generates an analog electrical signal corresponding to mechanical vibration. Although this analog electrical signal is not actually based on real sound, it can still be used as the picked-up "audio signal".

[0101] When the electronic device 100 acquires audio signals, it can acquire analog audio signals through the microphone 160, convert the analog audio signals into digital audio signals through the audio module 140, which may include amplifying the gain of the analog audio signals through an amplifier, filtering out high-frequency useless audio signals in the analog audio signals through a low-pass filter, and then converting the analog audio signals into digital audio signals through an analog-to-digital converter.

[0102] In some embodiments, because the microphone 160 includes a large number of nonlinear components such as diodes and amplifiers, the microphone 160 has nonlinear demodulation characteristics. This characteristic allows the microphone 160 to generate a low-frequency (e.g., below the upper limit of the pickup frequency) demodulated signal when it receives high-frequency (e.g., above the upper limit of the pickup frequency) sound. Therefore, please refer to... Figures 4-6 As shown, Figures 4-6 The dashed line in the image represents the upper limit of the pickup frequency. When electronic device 100 receives a signal such as... Figure 4 When the ultrasonic wave is transmitted, since the frequency of the ultrasonic wave is greater than the upper limit of the pickup frequency, the electronic device 100 can acquire the signal as shown in the figure, based on the aforementioned nonlinear demodulation characteristics. Figure 5 The ultrasonic waves and low-frequency harmonics shown are below the upper limit of the pickup frequency. After being processed by the audio module 140, the ultrasonic waves are filtered out, but the low-frequency harmonics are retained, thus becoming part of the acquired digital audio signal.

[0103] As mentioned above, due to the influence of the microphone's hardware structure, the audio signal collected by the microphone may include audio signals corresponding to the user's actual speech, as well as audio signals generated by laser attacks or ultrasonic attacks. Specifically, the audio signal generated by the microphone under laser attacks is the laser attack signal, and the audio signal generated by the microphone under ultrasonic attacks is the ultrasonic attack signal.

[0104] Taking the interaction between electronic devices and users as an example, under normal circumstances, users can issue voice commands, and electronic devices can receive these commands and perform corresponding operations based on the voice. However, if an attacker modulates the voice command onto a laser and shines the modulated laser onto a microphone, the microphone, based on the photoelectric and photoacoustic effects, will also pick up the voice command modulated into the laser. This allows the attacker to attack the electronic device via laser; this type of attack is called a "light commands" attack or laser attack. Alternatively, if the attacker modulates the voice command into ultrasound, based on the nonlinear demodulation characteristics of the microphone, the microphone may also pick up the voice command modulated into ultrasound, allowing the attacker to attack the electronic device via ultrasound; this type of attack is called an "ultrasound attack."

[0105] It can be seen that these abnormal audio signals generated by laser or ultrasonic attacks can seriously threaten the security of electronic devices and user data. For example, attackers can use light attacks to remotely control smart home devices and smart door locks to open or close, unlock and start vehicles, control smart speakers or mobile phones to make online shopping, and so on.

[0106] In some embodiments, electronic devices can add additional reminders or confirmation actions for the user to verify that the collected audio signal comes from a genuine user voice. For example, when detecting a voice command like "order 100 mobile phones," the electronic device can remind the user via a pop-up window that "you are ordering 100 mobile phones via voice shopping," or request the user to re-enter a password or verification code to confirm that the voice command comes from the user. Another example is the frequent appearance of a message on the user's phone screen: "Your phone is suspected of being attacked by a laser; please confirm if this is your operation." While these methods can prevent interference from abnormal audio signals to some extent, they sacrifice the convenience of the electronic device. Furthermore, cumbersome operations and false alarms can lead users to question the reliability of the electronic device and the brand, ultimately causing them to refuse to use or disable related functions, resulting in economic losses for the brand.

[0107] To at least partially solve the aforementioned technical problems, embodiments of this application provide a method for processing voice services. In this embodiment, when a first audio signal is acquired via a microphone, a second audio signal can be output based on the first audio signal. This ensures that, if the first audio signal includes a laser attack signal, the signal-to-noise ratio (SNR) of the second audio signal is greater than that of the first audio signal. Therefore, when processing voice services based on the second audio signal, the laser attack signal in the second audio signal is more difficult to identify, thus reducing the risk of laser attacks and improving the security of electronic devices and user data. Since no additional reminders or confirmations are required for the user, the convenience of the electronic device is not reduced, the user experience is improved, and economic losses to the company's brand are avoided.

[0108] The technical solutions of this application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0109] As described above, a microphone can sense sound and laser light in its environment (including acoustic-electric effect or photoelectric effect) to obtain corresponding electrical signals, i.e., analog audio signals. In some embodiments, the analog audio signal can be converted from analog to digital to obtain a corresponding digital audio signal; conversely, the digital audio signal can be converted from digital to analog to obtain a corresponding analog audio signal. Therefore, in the following embodiments, unless otherwise specified, each audio signal can be either a digital audio signal or an analog audio signal, and whether it is a digital or analog audio signal can be converted according to the needs of any step in the method for processing voice services provided in the embodiments of this application. For example, to facilitate the internal processing or storage of audio signals in electronic devices, analog audio signals can be converted to digital audio signals. As another example, before playing sound through a speaker, the electronic device can convert the digital audio signal to an analog audio signal, and then use the analog audio signal to drive the speaker to play the corresponding sound.

[0110] Please refer to Figure 7 This is a flowchart illustrating a method for processing voice services provided in an embodiment of this application. In this embodiment, noise is added to the audio signal used for voice services in the digital domain. It should be noted that this method does not rely on... Figure 7 The specific order described below is a limitation. It should be understood that in other embodiments, the order of some steps in the method may be interchanged according to actual needs, or some steps may be omitted or deleted.

[0111] The method includes the following steps:

[0112] S701, the electronic device acquires the first audio signal.

[0113] An electronic device can acquire a first audio signal via a microphone, which can be used to process voice services. In some embodiments, the first audio signal may include audio signals with frequencies lower than a preset pickup frequency limit (such as actual voice output by the user), i.e., normal audio signals. In other embodiments, the first audio signal may include abnormal audio signals such as laser attack signals generated by laser attacks and / or ultrasonic attack signals generated by ultrasonic attacks.

[0114] S702, the electronic device determines whether the operation of acquiring the first audio signal is interfered with. If so, it executes S703; otherwise, it executes S706.

[0115] As mentioned above, the first audio signal acquired by the electronic device may include normal audio signals or abnormal audio signals. Therefore, in order to reduce or avoid the risks that abnormal audio signals may pose to subsequent voice services, the electronic device can accurately select the appropriate processing method by determining whether the acquisition operation of the first audio signal is interfered with, thereby reducing unnecessary data processing volume and processing steps. If the acquisition operation is interfered with, the first audio signal may include abnormal audio signals; if the acquisition operation is not interfered with, the first audio signal will not include abnormal audio signals.

[0116] In some embodiments, the electronic device may perform at least one of laser attack detection and ultrasonic attack detection; if it is determined that it is subjected to either a laser attack or an ultrasonic attack, then it is determined that the audio signal acquisition operation is interfered with. In some embodiments, the electronic device may perform laser attack detection or ultrasonic attack detection based on the acquired first audio signal.

[0117] In some embodiments, the electronic device may input a first audio signal into a first machine learning model and obtain a detection result from the first machine learning model, which may be used to indicate whether the operation of the first audio signal includes laser attack signals and / or ultrasonic attack signals.

[0118] The electronic device can acquire a first sample set in advance and train a first machine learning model using this set. The first sample set includes multiple samples, each consisting of audio signals collected under conditions of interference from laser or ultrasonic attacks, and audio signals collected without such interference. Each sample is labeled based on whether it has been subjected to laser or ultrasonic attacks. Of course, in practical applications, the electronic device can also acquire a pre-trained first machine learning model from other electronic devices.

[0119] In some embodiments, the microphone is an array comprising multiple microphone units, and the first audio signal may include sub-signals from the multiple microphone units. Since the multiple microphone units are positioned differently, when an attacker attacks the microphone of an electronic device with a laser, they may attack some of the microphone units with a narrower laser beam or all of the microphone units with a wider laser beam. If the attacker attacks some of the microphone units with a narrower laser signal, the correlation between the multiple sub-signals included in the first audio signal will be very low because the other microphone units do not receive the laser. If the attacker attacks most of the microphone units with a wider laser beam or multiple narrower laser beams, the phase difference between the multiple sub-signals included in the first audio signal will be very small because the laser is essentially incident on each microphone unit from a vertical direction. Therefore, correlation detection and / or directionality detection can be performed on the multiple sub-signals included in the first audio signal to determine whether a laser attack has occurred. Taking correlation detection as an example, if the correlation between the multiple sub-signals included in the first audio signal is very low, such as a correlation coefficient less than a preset correlation coefficient threshold, then a laser attack can be determined. Taking directional detection as an example, if the phase difference between the multiple sub-signals included in the first audio signal is very small, such as the average phase difference being greater than a preset phase threshold, then it can be determined that the signal is under laser attack.

[0120] In some embodiments, the electronic device can determine whether the operation of acquiring the first audio signal is interfered with by laser attack or ultrasonic attack based on the difference between the base frequency of the normal audio signal and the base frequency of the abnormal audio signal.

[0121] It should be noted that in practical applications, electronic devices can also detect whether the operation of acquiring the first audio signal is interfered with by laser or ultrasonic attacks through other methods. This application does not limit the method of detecting whether the first audio signal is interfered with by laser or ultrasonic attacks. In some embodiments, the electronic device can detect whether the operation of acquiring the first audio signal is interfered with by laser or ultrasonic attacks based on one or more detection methods, such that the detection rate of interference is greater than a preset detection rate threshold, and the false detection rate is less than a preset false detection rate threshold, thereby ensuring the accuracy of detecting laser or ultrasonic attack interference. In some embodiments, the detection rate threshold can be 99%, and the false detection rate threshold can be 10 times / hour or 5 times / hour. Of course, in practical applications, the detection rate threshold and the false detection rate threshold can also be other values.

[0122] In some embodiments, the electronic device may determine the confidence level of the operation of acquiring the first audio signal being interfered with by a laser attack or an ultrasonic attack by one or more of the above detection methods. When the confidence level is greater than or equal to a preset confidence level threshold, it is determined that the operation of acquiring the first audio signal is interfered with by a laser attack or an ultrasonic attack. When the confidence level is less than the confidence level threshold, it is determined that the operation of acquiring the first audio signal is not interfered with by a laser attack or an ultrasonic attack.

[0123] In some embodiments, since the audio signal is modulated into the optical signal when the microphone is attacked by a laser, and the audio signal is modulated into the ultrasonic wave when the microphone is attacked by an ultrasonic wave, and the energy of the optical signal, which is used as a carrier signal, is more stable than the energy of the ultrasonic wave, which is also used as a carrier signal, the electronic device can acquire the signal energy of the first audio signal and determine whether the interference to the operation of acquiring the first audio signal is a laser attack or an ultrasonic attack based on the signal energy distribution of the first audio signal.

[0124] In some embodiments, the electronic device can acquire the RMS of a first audio signal. If the amplitude RMS is less than a preset first RMS threshold, it is determined that the interference affecting the acquisition of the first audio signal is a laser attack; otherwise, it is determined that the interference affecting the acquisition of the first audio signal is specifically an ultrasonic attack. In some embodiments, the first RMS threshold can be -50 dB. Of course, in practical applications, the first RMS threshold can also be other values, and this application embodiment does not limit the value of the first RMS threshold.

[0125] In some embodiments, the electronic device can divide the first audio signal into multiple frames according to time sequence (e.g., each 100ms segment of the first audio signal is considered one frame), determine the RMS amplitude of each frame, and then determine a first difference range of the RMS amplitudes of the multiple frames. If the first difference range is less than a preset second RMS threshold, it is determined that the interference received by the operation of acquiring the first audio signal is a laser attack; otherwise, it is determined that the interference received by the operation of acquiring the first audio signal is an ultrasonic attack. In some embodiments, the second RMS threshold can be 3dB. Of course, in practical applications, the second RMS threshold can also be other values, and this application embodiment does not limit the value of the second RMS threshold.

[0126] In some embodiments, the electronic device can acquire a set of audio signals within a preset duration, the set of audio signals including a first audio signal and other audio signals acquired by a microphone. The preset duration can be 5s-10s or other durations. Interference experienced by the microphone within this preset duration may very well be the same interference event. Furthermore, if the interference is a laser attack, it may use the same laser as a carrier wave. Therefore, the electronic device can acquire a second difference range of the RMS amplitude of each frame in the audio signal set. If the second difference range is less than a preset second RMS threshold, it is determined that the interference experienced by the operation of acquiring the first audio signal is a laser attack; otherwise, it is determined that the interference experienced by the operation of acquiring the first audio signal is an ultrasonic attack.

[0127] In some embodiments, the electronic device can also determine whether the operation of acquiring the eighth audio signal is interfered with; if so, it executes S703; otherwise, it executes S706. The eighth audio signal can be an audio signal acquired before the first audio signal. That is, if the electronic device determines that the operation of acquiring the eighth audio signal has been interfered with by laser or ultrasonic attacks before acquiring the first audio signal, then the electronic device can determine that the current operation of acquiring the first audio signal has also been interfered with in the same way. It should be noted that the method by which the electronic device determines whether the operation of acquiring the eighth audio signal is interfered with can be the same as or similar to the method by which the electronic device determines whether the operation of acquiring the first audio signal is interfered with.

[0128] It should be noted that S702 is an optional step. When S702 is omitted, the electronic device can execute S703 after S701.

[0129] S703, the electronic device acquires a third audio signal.

[0130] The third audio signal can be a noise signal. In some embodiments, the third audio signal can be white noise. Of course, in practical applications, the third audio signal can also be other types of noise signals, such as additive noise or multiplicative noise.

[0131] In some embodiments, the electronic device may acquire a stored third audio signal. Alternatively, in other embodiments, the electronic device may generate the third audio signal using hardware and / or software for generating noise signals.

[0132] In some embodiments, since the frequency range of human speech is between 300Hz and 8kHz, the frequency range of the third audio signal can include at least a portion of the frequencies between 0Hz and 16kHz. In some embodiments, the frequency range of the third audio signal can be 0Hz-8kHz, 0Hz-16kHz, 300Hz-8kHz, or 300Hz-16kHz, thereby interfering with any human voice that may be included in the first audio signal. Therefore, when an attacker attacks the microphone of an electronic device using lasers or ultrasound, the third audio signal can interfere with the laser attack signal and / or ultrasound signal included in the first audio signal, reducing the harm of the laser attack signal and / or ultrasound signal to subsequent voice services.

[0133] In some embodiments, since a higher power in the third audio signal results in better interference with the first audio signal, the power of the third audio signal can be positively correlated with the likelihood that the operation of acquiring the first audio signal is interfered with. That is, the greater the likelihood that the operation of acquiring the first audio signal is interfered with, the stronger the interference capability of the acquired third audio signal.

[0134] In some embodiments, the power of the third audio signal is powr = f(prob), where f() is an increasing function. In some embodiments, f() can be tanh(), and prob is the confidence level that the operation of acquiring the first audio signal is interfered with.

[0135] S704, the electronic device superimposes the first audio signal and the third audio signal in the digital domain to obtain the second audio signal.

[0136] Electronic devices can superimpose a first audio signal and a third audio signal in the digital domain. This means the third audio signal interferes with the first audio signal in the digital domain, resulting in a second audio signal that includes noise in addition to one or more of the following: normal audio signals, laser attack signals, and ultrasonic attack signals. Therefore, the signal-to-noise ratio (SNR) of the second audio signal is greater than that of the first audio signal. Since normal audio signals are obtained by a microphone from sounds with frequencies below the upper limit of the pickup frequency, while abnormal audio signals are obtained by demodulating ultrasonic waves and / or lasers, normal audio signals are more robust than abnormal audio signals. Therefore, the noise signal interferes less with normal audio signals but significantly interferes with laser or ultrasonic attack signals, thus reducing the risk of laser or ultrasonic attacks on electronic devices and improving the security of electronic devices and user data. Furthermore, because the interference occurs in the digital domain, this process is imperceptible to the user, enhancing the user experience.

[0137] The signal-to-noise ratio (SNR) refers to the ratio of signal to noise that the electronic system needs to process. In this embodiment, the higher the SNR of the audio signal, the better the signal quality of the audio signal, and the higher the accuracy of the voice information carried by the audio signal in the subsequent voice service.

[0138] In some embodiments, if both the first audio signal and the third audio signal are digital audio signals, they can be directly superimposed in the digital domain. In other embodiments, if either the first audio signal or the third audio signal is an analog audio signal, the first or third audio signal, which belongs to the analog domain, can be converted into a digital domain audio signal, and then the first and third audio signals, which both belong to the digital domain, can be superimposed in the digital domain.

[0139] S705, electronic equipment based on second audio signal processing voice services.

[0140] Because the second audio signal, obtained after noise enhancement, has a higher signal-to-noise ratio than the first audio signal captured by the microphone, and because normal audio signals are more robust than abnormal audio signals, when processing voice services based on the second audio signal, even if the voice information that may be included in the ultrasonic or laser attack signal has been interfered with and is unrecognizable, the voice information that may be included in the normal audio signal can still be well identified, thus ensuring the security of electronic devices and user data. For example, if the second audio signal includes a false voice command "order 100 mobile phones" generated by a laser or ultrasonic attack, the electronic device will be unable to detect the voice command in the second audio signal through the VAD after the noise interference, and / or will be unable to identify the specific content of the voice command through KWS or ASR, and therefore will be unable to execute the voice command.

[0141] In addition, the method by which electronic devices process voice services based on the second audio signal can be referred to as follows: Figure 8 The detailed description is in the text.

[0142] S706, electronic equipment based on first audio signal processing voice services.

[0143] The electronic device determines whether the acquisition of the first audio signal is interfered with. If so, it outputs the second audio signal based on the first signal and processes voice services based on the second audio signal. Otherwise, it processes voice services based on the first audio signal and no longer outputs the second audio signal, thereby improving the accuracy of noise addition and reducing interference with the normal interaction between the electronic device and the user.

[0144] It should be noted that the way electronic devices process voice services based on the first audio signal can be the same as or similar to the way S705 processes voice services based on the second audio signal, and will not be elaborated here.

[0145] In this embodiment, when a first audio signal is acquired via a microphone, a third audio signal can be obtained. The third audio signal and the first audio signal are then superimposed in the digital domain to obtain a second audio signal. This allows the signal-to-noise ratio (SNR) of the second audio signal to be higher than that of the first audio signal when the first audio signal includes laser or ultrasonic attack signals. Therefore, when processing voice services based on the second audio signal, laser or ultrasonic attack signals in the second audio signal are more difficult to identify, thus reducing the risk of laser or ultrasonic attacks and improving the security of electronic devices and user data. Since no additional reminders or confirmations are required for the user, the convenience of the electronic device is not reduced, the user experience is improved, and economic losses to the company's brand are avoided. Furthermore, because this interference process occurs in the digital domain, it is not perceived by the user, further enhancing the user experience.

[0146] Please refer to Figure 9 This is a flowchart illustrating a method for processing voice services provided in an embodiment of this application. In this embodiment, noise is added to the audio signal used for voice services in the analog domain. It should be noted that this method does not rely on... Figure 9 The specific order described below is a limitation. It should be understood that in other embodiments, the order of some steps in the method may be interchanged according to actual needs, or some steps may be omitted or deleted.

[0147] The method includes the following steps:

[0148] S901, the electronic device acquires the first audio signal.

[0149] The method by which the electronic device acquires the first audio signal can be found in the relevant description in S701 above, and will not be repeated here.

[0150] S902, the electronic device determines whether the operation of acquiring the first audio signal is interfered with. If so, it executes S903; otherwise, it executes S906.

[0151] The method by which the electronic device determines whether the operation of acquiring the first audio signal is interfered with can be found in the relevant description in S702 above, and will not be repeated here.

[0152] In some embodiments, the electronic device may also determine whether the acquisition of the eighth audio signal is interfered with; if so, it executes S703; otherwise, it executes S706. The eighth audio signal can be an audio signal acquired before the first audio signal. It should be noted that the method by which the electronic device determines whether the acquisition of the eighth audio signal is interfered with can be the same as or similar to the method by which the electronic device determines whether the acquisition of the first audio signal is interfered with.

[0153] It should be noted that S902 is an optional step. When S902 is omitted, the electronic device can execute S903 after S901. That is, the electronic device can execute S903 as soon as the first audio signal is detected, instead of executing S902 to determine whether the operation of acquiring the first audio signal is interfered with.

[0154] S903, the electronic device acquires a third audio signal.

[0155] The third audio signal can be a noise signal. In some embodiments, the third audio signal can be white noise. Of course, in practical applications, the third audio signal can also be other types of noise signals.

[0156] In some embodiments, the frequency range of the third audio signal may include at least a portion of the frequencies from 0Hz to 16kHz. In some embodiments, the frequency range of the third audio signal may be 0Hz-8kHz, 0Hz-16kHz, 300Hz-8kHz, or 300Hz-16kHz.

[0157] It should be noted that the method by which electronic devices acquire third audio signals can also refer to the relevant description in the aforementioned S703, and will not be repeated here.

[0158] S904, the electronic device plays sound based on a third audio signal and acquires a second audio signal through a microphone, wherein the duration of the played sound and the duration of the acquired second audio signal at least partially overlap.

[0159] Since sound waves in nature belong to the analog domain, when an electronic device plays sound based on a third audio signal, and the duration of playing the sound at least partially overlaps with the duration of the collected sound, the second audio signal can be obtained by the microphone. That is, the first audio signal is noise-added in the analog domain by the third audio signal, so that the second audio signal includes one or more of the normal audio signal, laser attack signal, and ultrasonic attack signal, as well as the noise signal.

[0160] In some embodiments, the electronic device may play a sound corresponding to a third audio signal through a speaker and acquire a second audio signal through a microphone, wherein the duration of playing the sound corresponding to the third audio signal at least partially overlaps with the duration of acquiring the second audio signal.

[0161] In other embodiments, in order to reduce the disturbance to the user caused by playing the first analog audio signal, the electronic device can modulate the third audio signal to the fourth audio signal to obtain the fifth audio signal. The frequencies of the fourth and fifth audio signals are greater than 16 kHz. The sound corresponding to the fifth audio signal is played through a speaker, and the second audio signal is acquired through a microphone. The duration of playing the sound corresponding to the fifth audio signal at least partially overlaps with the duration of the second audio signal.

[0162] The fourth audio signal can be a carrier signal, used as a carrier for another signal to be transmitted; the third audio signal can be a modulated signal, which is another signal to be transmitted; and the fifth audio signal is a modulated signal.

[0163] In some embodiments, the frequency range of the fourth and fifth audio signals may include 24 kHz and / or 36 kHz. Of course, in practical applications, the frequency range of the fourth and fifth audio signals may include other frequencies.

[0164] In some embodiments, the duration for which the electronic device plays sound based on a third audio signal can be 10 seconds or 15 seconds. Of course, in practical applications, this duration can also be other values, and this application embodiment does not limit the size of the duration.

[0165] S905, electronic equipment based on second audio signal processing voice services.

[0166] S906, electronic equipment based on first audio signal processing voice service.

[0167] It should be noted that the way electronic devices execute S905-S906 can be the same as or similar to the way they execute S705 and S706, which will not be elaborated here.

[0168] In this embodiment, when a first audio signal is acquired via a microphone, sound can be played based on a third audio signal, and a second audio signal can be acquired via the microphone. The duration of the played sound at least partially overlaps with the duration of the acquired second audio signal, thereby adding noise in the analog domain. This allows the signal-to-noise ratio (SNR) of the second audio signal to be higher than that of the first audio signal when the first audio signal includes laser or ultrasonic attack signals. Therefore, when processing voice services based on the second audio signal, laser or ultrasonic attack signals in the second audio signal are more difficult to identify, thus reducing the risk of laser or ultrasonic attacks and improving the security of electronic devices and user data. Since no additional reminders or confirmations are required for the user, the convenience of the electronic device is not reduced, the user experience is improved, and economic losses to the company's brand are avoided. Furthermore, the third audio signal can be modulated to a fourth audio signal at a frequency above 16kHz to obtain a fifth audio signal. Playing a sound corresponding to the fifth audio signal through a speaker makes the played sound less perceptible to the user, further enhancing the user experience.

[0169] It should be noted that in some embodiments, the electronic device may also determine the timing of playing sound based on the third audio signal in other ways, such that the duration of playing sound based on the third audio signal at least partially overlaps with the duration of the second audio signal acquired by the microphone. For example, the electronic device may continuously execute S903-S904 to play sound based on the third audio signal. In this case, S901 and S902 can be omitted.

[0170] Please refer to Figure 10 This is a flowchart illustrating a method for processing voice services provided in an embodiment of this application. In this embodiment, noise is added to the audio signal used for voice services in both the digital and analog domains. It should be noted that this method does not rely on... Figure 10 The specific order described below is a limitation. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0171] S1001, the electronic device acquires the first audio signal.

[0172] S1002, the electronic device determines whether the operation of acquiring the first audio signal is interfered with. If so, it executes S1003 and S1005; otherwise, it executes S1008.

[0173] It should be noted that S1002 is an optional step. When the electronic device omits S1002, S1003 and S1005 can be executed after S1001.

[0174] S1003, the electronic device acquires the third audio signal.

[0175] It should be noted that the way the electronic device executes S1001-S1003 can be the same as or similar to the way it executes S701-S703, which will not be elaborated here.

[0176] S1004, the electronic device plays sound based on the third audio signal and acquires the second audio signal through a microphone, wherein the duration of the played sound and the duration of the acquired second audio signal at least partially overlap.

[0177] It should be noted that the way the electronic device executes S1004 can be the same as or similar to the way it executes S904, which will not be elaborated here.

[0178] S1005, the electronic device acquires the sixth audio signal.

[0179] The sixth audio signal can be a noise signal with a frequency range including at least a portion of the frequencies from 0Hz to 16kHz. In some embodiments, the sixth audio signal and the third audio signal are the same audio signal; in other embodiments, the sixth audio signal and the third audio signal are different audio signals. When the sixth audio signal and the third audio signal are the same audio signal, the electronic device can add noise to the first audio signal in both the analog and digital domains using the same audio signal; when the sixth audio signal and the third audio signal are different audio signals, the electronic device can add noise to the first audio signal in both the analog and digital domains using different audio signals.

[0180] S1006, the electronic device superimposes the sixth audio signal and the second audio signal acquired by the microphone in the digital domain to obtain a new second audio signal.

[0181] The way in which the electronic device executes S1005-S1006 can be the same as or similar to the way in which it executes S703-S704, and will not be described in detail here.

[0182] S1007, electronic equipment based on new second audio signal processing voice service.

[0183] The new second audio signal includes a noise signal in addition to one or more of the following: normal audio signal, laser attack signal, and ultrasonic attack signal.

[0184] S1008, Electronic device based on first audio signal processing voice service.

[0185] It should be noted that the way electronic devices execute S1007-S1008 can be the same as or similar to the way they execute S705 and S706, which will not be elaborated here.

[0186] In this embodiment, when a first audio signal is acquired via a microphone, sound can be played based on a third audio signal, and a second audio signal can be acquired via the microphone. The duration of the played sound at least partially overlaps with the duration of the acquired second audio signal. A sixth audio signal can also be acquired. The sixth audio signal and the second audio signal acquired via the microphone are superimposed in the digital domain to obtain a new second audio signal. This simultaneously adds noise in both the analog and digital domains, ensuring that the signal-to-noise ratio (SNR) of the second audio signal is higher than that of the first audio signal when the first audio signal includes laser or ultrasonic attack signals. Therefore, when processing voice services based on the new second audio signal, laser or ultrasonic attack signals in the new second audio signal are more difficult to identify, thus reducing the risk of laser or ultrasonic attacks and improving the security of electronic devices and user data. Since no additional reminders or confirmations are required for the user, the convenience of the electronic device is not reduced, the user experience is improved, and economic losses to the company's brand are avoided.

[0187] It should be noted that in some embodiments, the electronic device may also determine the timing of playing sound based on the third audio signal in other ways, such that the duration of playing sound based on the third audio signal at least partially overlaps with the duration of the second audio signal acquired by the microphone. For example, the electronic device may continuously execute S1003-S1004 to play sound based on the third audio signal. In this case, S1001 and S1002 can be omitted.

[0188] Please refer to Figure 11 This is a flowchart illustrating a method for processing voice services provided in an embodiment of this application. In this embodiment, when the microphone of an electronic device is attacked by a laser, noise is added to the audio signal used for voice services in the analog domain using ultrasonic waves. It should be noted that this method does not rely on... Figure 11 The specific order described below is a limitation. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0189] S1101, the electronic device acquires the first audio signal.

[0190] S1102, the electronic device determines whether the operation of acquiring the first audio signal is under laser attack. If so, it executes S1103; otherwise, it executes S1107.

[0191] It should be noted that S1102 is an optional step. When the electronic device omits S1102, S1103 can be executed after S1101.

[0192] S1103, the electronic device acquires the third audio signal.

[0193] It should be noted that the way the electronic device executes S1101-S1103 can be the same as or similar to the way it executes S701-S703, which will not be elaborated here.

[0194] S1104, the electronic device modulates the third audio signal to the fourth audio signal to obtain the fifth audio signal.

[0195] Among them, the frequencies of the fourth and fifth audio signals are greater than 16kHz.

[0196] S1105, the electronic device plays the sound corresponding to the fifth audio signal through a speaker and acquires the second audio signal through a microphone.

[0197] The duration of the sound corresponding to the fifth audio signal played by the electronic device through the speaker at least partially overlaps with the duration of the second audio signal acquired by the microphone.

[0198] It should be noted that the manner in which the electronic device executes S1104 and S1105 can be found in the relevant description in S904, and will not be repeated here.

[0199] S1106, Electronic equipment based on second audio signal processing voice service.

[0200] The new second audio signal includes a noise signal in addition to one or more of the following: normal audio signal, laser attack signal, and ultrasonic attack signal.

[0201] S1107, Electronic device based on first audio signal processing semantic service.

[0202] It should be noted that the way the electronic device executes S1106-S1107 can be the same as or similar to the way it executes S705 and S706, which will not be elaborated here.

[0203] In this embodiment, when the first audio signal acquired via microphone is subjected to a laser attack, a third audio signal can be obtained. This third audio signal is then modulated onto a fourth audio signal to obtain a fifth audio signal. The sound corresponding to the fifth audio signal is played through a speaker, and a second audio signal is acquired via microphone. Noise is added in the analog domain so that when the first audio signal includes a laser attack signal, the signal-to-noise ratio (SNR) of the second audio signal can be higher than that of the first audio signal. Therefore, when processing voice services based on the second audio signal, the laser attack signal in the second audio signal is more difficult to identify, thus reducing the risk of laser attacks and improving the security of electronic devices and user data. Since no additional reminders or confirmations are required for the user, the convenience of the electronic device is not reduced, the user experience is improved, and economic losses to the company's brand are avoided. Furthermore, because the frequency of the fifth audio signal is greater than 16kHz, the played sound is less perceptible to the user, further enhancing the user experience.

[0204] It should be noted that in some embodiments, the electronic device may also determine the timing of playing the sound corresponding to the fifth audio signal through the speaker in other ways, such that the duration of playing the sound corresponding to the fifth audio signal through the speaker at least partially overlaps with the duration of the second audio signal acquired by the microphone. For example, the electronic device may continuously execute S1103-S1105 to play the sound corresponding to the fifth audio signal. In this case, S1101 and S1102 can be omitted.

[0205] In the above embodiments, when it is determined that the microphone is under laser or ultrasonic attack, the electronic device can add noise to the first audio signal acquired by the microphone in at least one aspect of the digital and analog domains, thereby outputting a second audio signal with a higher signal-to-noise ratio than the first audio signal. The audio signal used for noise addition is the same regardless of whether the attack is laser or ultrasonic. However, since a high laser power could burn out the microphone when attacked by laser, the laser power used in laser attacks is relatively low. Therefore, in another embodiment, when it is determined that the microphone is under laser attack, the electronic device can add noise to the first audio signal based on a third audio signal (and a sixth audio signal); and when it is determined that the microphone is under ultrasonic attack, it can add noise to the first audio signal based on a seventh audio signal (and a ninth audio signal). The power of the seventh audio signal can be greater than the power of the third audio signal, and the power of the ninth audio signal can be greater than the power of the sixth audio signal.

[0206] It should be noted that the way an electronic device adds noise to the first audio signal based on the seventh audio signal can be the same as or similar to the way an electronic device adds noise to the first audio signal based on the third audio signal; the way an electronic device adds noise to the first audio signal based on the ninth audio signal can be the same as or similar to the way an electronic device adds noise to the first audio signal based on the sixth audio signal.

[0207] Please refer to Figure 12 This is a flowchart illustrating a method for processing voice services provided in an embodiment of this application. It should be noted that this method is not based on... Figure 12 The specific order described below is a limitation. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0208] S1201, the electronic device acquires the first audio signal for voice services via a microphone.

[0209] The method by which the electronic device collects the first audio signal for voice services through the microphone can be the same as or similar to the method by which the electronic device collects the first audio signal in the aforementioned S701, and will not be described in detail here.

[0210] S1202, the electronic device outputs a second audio signal based on a first audio signal, wherein if the first audio signal includes a laser attack signal, the signal-to-noise ratio of the second audio signal is greater than the signal-to-noise ratio of the first audio signal.

[0211] In some embodiments, the electronic device may output a second audio signal based on the first audio signal by performing the same or similar manner as described in one or more steps of S702-S704 above.

[0212] In some embodiments, the electronic device may output a second audio signal based on a first audio signal by performing the same or similar manner as described in one or more steps of S902-S904 above.

[0213] In some embodiments, the electronic device may output a second audio signal based on the first audio signal by performing the same or similar manner as described in one or more steps of S1002-S1006 above.

[0214] In some embodiments, the electronic device may output a second audio signal based on a first audio signal by performing the same or similar steps as described in one or more of the steps in S1102-S1105 above.

[0215] Of course, in practical applications, electronic devices can also output a second audio signal based on the first audio signal in other ways, as long as it is ensured that the signal-to-noise ratio of the second audio signal is greater than that of the first audio signal when the first audio signal includes the laser attack signal.

[0216] S1203, Electronic equipment based on second audio signal processing voice service.

[0217] It should be noted that the way the electronic device processes voice services based on the second audio signal can be the same as or similar to the way it processes voice services based on the second audio signal in the aforementioned S705, and will not be described in detail here.

[0218] In this embodiment, when a first audio signal is acquired via a microphone, a second audio signal can be output based on the first audio signal. This results in a higher signal-to-noise ratio (SNR) for the second audio signal compared to the first audio signal, even if the first audio signal includes a laser attack signal. Therefore, when processing voice services based on the second audio signal, the laser attack signal in the second audio signal is more difficult to identify, thus reducing the risk of laser attacks and improving the security of electronic devices and user data. Since no additional reminders or confirmations are required for the user, the convenience of the electronic device is not reduced, the user experience is improved, and economic losses to the company's brand are avoided.

[0219] Please refer to Figure 8 This is a flowchart illustrating a method for processing voice services provided in an embodiment of this application. The following will use S705 as an example to describe how an electronic device processes voice services. It should be noted that this method does not rely on... Figure 8 The specific order described below is a limitation. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0220] S801, the electronic device performs VAD detection on the second audio signal to obtain the tenth audio signal, which includes voice information.

[0221] The electronic device can perform VAD detection on the second audio signal to determine the start and end points of the included speech information, and define the audio signal between the start and end points as the tenth audio signal. That is, the tenth audio signal can be at least a part of the second audio signal.

[0222] It should be noted that if the electronic device fails to detect voice information from the second audio signal, that is, fails to obtain the tenth audio signal, it can stop executing the subsequent steps.

[0223] S802, the electronic device performs KWS detection on the tenth audio signal.

[0224] Although the tenth audio signal includes voice information, this voice information may contain a large amount of information, and only some of this information may be related to the electronic device, such as triggering certain functions of the electronic device. Therefore, in order to improve the accuracy and efficiency of voice service processing, the electronic device can perform KWS detection on the tenth audio signal to determine whether the audio signal contains predefined words or phrases.

[0225] For example, if the electronic device is a smart speaker and the predefined word is the wake word "Xiaoyi" that the user has set for the smart speaker, then when the electronic device is in sleep mode, it will obtain the tenth audio signal and perform KWS detection on the tenth audio signal. If the tenth audio signal includes "Xiaoyi", it will switch to wake-up mode. If the tenth audio signal does not include "Xiaoyi", it will continue to remain in sleep mode.

[0226] S803, the electronic device performs ASR processing on the tenth audio signal.

[0227] Electronic devices can perform ASR processing on the tenth audio signal to convert the voice information included in the tenth audio signal into text information. This text information can be used by electronic devices to further accurately determine the user's intention or instruction, thereby enabling them to engage in dialogue with the user or perform corresponding operations according to the user's intention or instruction.

[0228] Taking a smart speaker as an example, the smart speaker has entered the wake-up state through S802. The smart speaker can then perform ASR processing on the voice included in the tenth audio signal, and the resulting text information is "turn on the TV". Therefore, the smart speaker can send an open command to the smart TV to turn on the smart TV.

[0229] Based on the same concept, as an implementation of the above method, this application provides an apparatus for processing voice services. This apparatus embodiment corresponds to the aforementioned method embodiment. For ease of reading, this apparatus embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the apparatus in this embodiment can correspondingly implement all the contents of the aforementioned method embodiment.

[0230] Please refer to Figure 13 This is a schematic diagram of the structure of the device 1300 for processing voice services provided in an embodiment of this application. Figure 13 As shown, the device provided in this embodiment includes an audio module 140 and a voice service module 170.

[0231] The audio module 140 is used to acquire a first audio signal for voice services through a microphone and output a second audio signal based on the first audio signal. If the first audio signal includes a laser attack signal, the signal-to-noise ratio of the second audio signal is greater than that of the first audio signal.

[0232] Voice service module 170 is used for processing voice services based on the second audio signal.

[0233] In some embodiments, the audio module 140 is specifically used for:

[0234] Acquire a third audio signal, which is a noise signal with a frequency range including at least a portion of the frequencies from 0Hz to 16kHz;

[0235] Based on the third audio signal, the first audio signal is subjected to noise addition processing to obtain the second audio signal.

[0236] In some embodiments, the audio module 140 is specifically used for:

[0237] The sound corresponding to the third audio signal is played through a speaker, and the second audio signal is acquired through a microphone, wherein the duration of playing the sound corresponding to the third audio signal at least partially overlaps with the duration of acquiring the second audio signal.

[0238] In some embodiments, the audio module 140 is specifically used for:

[0239] The third audio signal is modulated onto the fourth audio signal to obtain the fifth audio signal. The frequencies of the fourth and fifth audio signals are greater than 16kHz.

[0240] The sound corresponding to the fifth audio signal is played through a speaker, and the second audio signal is acquired through a microphone, wherein the duration of playing the sound corresponding to the fifth audio signal and the duration of acquiring the second audio signal at least partially overlap.

[0241] In some embodiments, the audio module 140 is specifically used for:

[0242] The sixth audio signal and the second audio signal acquired by the microphone are superimposed in the digital domain to obtain a new second audio signal. The sixth audio signal is a noise signal with a frequency range including at least some frequencies in the range of 0Hz-16KHz.

[0243] Voice service module 170 is specifically used for:

[0244] Based on the new second audio signal processing voice service.

[0245] In some embodiments, the audio module 140 is specifically used for:

[0246] The third audio signal and the first audio signal are superimposed in the digital domain to obtain the second audio signal.

[0247] In some embodiments, the device further includes a processing module;

[0248] If the processing module determines that the microphone is under laser attack, it triggers the audio module 140 to add noise to the first audio signal based on the third audio signal.

[0249] In some embodiments, if the processing module determines that the microphone is under ultrasonic attack, it triggers the audio module 140 to add noise to the first audio signal based on the seventh audio signal, wherein the power of the seventh audio signal is greater than the power of the third audio signal.

[0250] In some embodiments, the processing module may be located in the aforementioned processor 110.

[0251] In some embodiments, the power of the third audio signal is positively correlated with the credibility of being subjected to a laser attack or an ultrasonic attack.

[0252] Based on the same concept, this application also provides an electronic device, which may include: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method described in the above method embodiments when the computer program is invoked.

[0253] The electronic device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so they will not be described again here.

[0254] Based on the same concept, this application also provides a chip system. The chip system includes a processor coupled to a memory, the processor executing a computer program stored in the memory to implement the methods described in the above-described method embodiments.

[0255] The chip system can be a single chip or a chip module composed of multiple chips.

[0256] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the methods described in the above-described method embodiments.

[0257] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to implement the method described in the above-described method embodiments.

[0258] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include at least: any entity or device capable of carrying computer program code to a photographic device / terminal device, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0259] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0260] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0261] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0262] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0263] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0264] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0265] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0266] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0267] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for processing voice services, characterized in that, The method includes: The first audio signal for voice services is acquired via a microphone; A second audio signal is output based on the first audio signal, wherein if the first audio signal includes a laser attack signal, the signal-to-noise ratio of the second audio signal is greater than that of the first audio signal; The voice service is processed based on the second audio signal.

2. The method according to claim 1, characterized in that, The step of outputting a second audio signal based on the first audio signal includes: Acquire a third audio signal, wherein the third audio signal is a noise signal with a frequency range including at least a portion of frequencies from 0Hz to 16KHz; Based on the third audio signal, the first audio signal is subjected to noise addition processing to obtain the second audio signal.

3. The method according to claim 2, characterized in that, The noise-adding process on the first audio signal based on the third audio signal includes: The sound corresponding to the third audio signal is played through a speaker, and the second audio signal is acquired through a microphone, wherein the duration of playing the sound corresponding to the third audio signal at least partially overlaps with the duration of acquiring the second audio signal.

4. The method according to claim 2, characterized in that, The noise-adding process on the first audio signal based on the third audio signal includes: The third audio signal is modulated onto the fourth audio signal to obtain the fifth audio signal, wherein the frequencies of the fourth audio signal and the fifth audio signal are greater than 16KHz; The sound corresponding to the fifth audio signal is played through a speaker, and the second audio signal is acquired through the microphone, wherein the duration of playing the sound corresponding to the fifth audio signal at least partially overlaps with the duration of acquiring the second audio signal.

5. The method according to claim 3 or 4, characterized in that, The method further includes: The sixth audio signal and the second audio signal acquired by the microphone are superimposed in the digital domain to obtain a new second audio signal. The sixth audio signal is a noise signal with a frequency range including at least some frequencies in the range of 0Hz-16KHz. The process of processing the voice service based on the second audio signal includes: The voice service is processed based on the new second audio signal.

6. The method according to claim 2, characterized in that, The noise-adding process on the first audio signal based on the third audio signal includes: The third audio signal and the first audio signal are superimposed in the digital domain to obtain the second audio signal.

7. The method according to any one of claims 2-6, characterized in that, The noise-adding process on the first audio signal based on the third audio signal includes: If it is determined that the microphone is under laser attack, then the first audio signal is noise-added based on the third audio signal.

8. The method according to claim 7, characterized in that, The method further includes: If it is determined that the microphone is under ultrasonic attack, then the first audio signal is subjected to noise enhancement based on the seventh audio signal, wherein the power of the seventh audio signal is greater than the power of the third audio signal.

9. The method according to any one of claims 2-6, characterized in that, The power of the third audio signal is positively correlated with the credibility of being subjected to the laser attack or ultrasonic attack.

10. An apparatus for processing voice services, characterized in that, The device includes an audio module and a voice service module; The audio module is used to acquire a first audio signal for voice services through a microphone and output a second audio signal based on the first audio signal. If the first audio signal includes a laser attack signal, the signal-to-noise ratio of the second audio signal is greater than that of the first audio signal. The voice service module is used to process the voice service based on the second audio signal.

11. The apparatus according to claim 10, characterized in that, The audio module is specifically used for: Acquire a third audio signal, wherein the third audio signal is a noise signal with a frequency range including at least a portion of frequencies from 0Hz to 16KHz; Based on the third audio signal, the first audio signal is subjected to noise addition processing to obtain the second audio signal.

12. The apparatus according to claim 11, characterized in that, The audio module is specifically used for: The sound corresponding to the third audio signal is played through a speaker, and the second audio signal is acquired through a microphone, wherein the duration of playing the sound corresponding to the third audio signal at least partially overlaps with the duration of acquiring the second audio signal.

13. The apparatus according to claim 11, characterized in that, The audio module is specifically used for: The third audio signal is modulated onto the fourth audio signal to obtain the fifth audio signal, wherein the frequencies of the fourth audio signal and the fifth audio signal are greater than 16KHz; The sound corresponding to the fifth audio signal is played through a speaker, and the second audio signal is acquired through the microphone, wherein the duration of playing the sound corresponding to the fifth audio signal at least partially overlaps with the duration of acquiring the second audio signal.

14. The apparatus according to claim 12 or 13, characterized in that, The audio module is specifically used for: The sixth audio signal and the second audio signal acquired by the microphone are superimposed in the digital domain to obtain a new second audio signal. The sixth audio signal is a noise signal with a frequency range including at least some frequencies in the range of 0Hz-16KHz. The voice service module is specifically used for: The voice service is processed based on the new second audio signal.

15. The apparatus according to claim 11, characterized in that, The audio module is specifically used for: The third audio signal and the first audio signal are superimposed in the digital domain to obtain the second audio signal.

16. The apparatus according to any one of claims 11, 12, 13, and 15, characterized in that, The device also includes a processing module: If the processing module determines that the microphone is under laser attack, it triggers the audio module to add noise to the first audio signal based on the third audio signal.

17. The apparatus according to claim 16, characterized in that, If the processing module determines that the microphone is under ultrasonic attack, it triggers the audio module to add noise to the first audio signal based on a seventh audio signal, wherein the power of the seventh audio signal is greater than the power of the third audio signal.

18. An electronic device, characterized in that, include: A memory and a processor, the memory being used to store a computer program; the processor being used to execute the method as described in any one of claims 1-9 when the computer program is invoked.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-9.

Images