A method and electronic device for preventing light wave attacks

Abstract

This application provides a method and electronic device for preventing light wave attacks. The method is applied to an electronic device comprising n microphones. The method includes: acquiring an audio signal containing a wake-up word through u of the n microphones; if the energy value corresponding to at least one of the u microphones deviates from a reference energy value of the n microphones within a preset range, the electronic device outputs an alarm message and is not woken up by the audio signal; if the energy value corresponding to each of the u microphones deviates from the reference energy value of the n microphones within a preset range, the electronic device is woken up by the audio signal; where n is a positive integer greater than or equal to 1; u is a positive integer greater than or equal to 1 and less than n. The method provided in this application can reduce the security risks caused by attacks on electronic devices from various light waves, including lasers.

Description

Technical Field

[0001] This application relates to the field of equipment security technology, and in particular to a method and electronic device for preventing light wave attacks. Background Technology

[0002] With the development of technology, electronic devices with voice control functions have gradually entered people's lives, providing great convenience. Electronic devices are generally equipped with multiple microphones (such as microphones) to collect the user's voice, analyze and process it to obtain specific control commands, and then execute those commands. Users can control electronic devices or their accessories via voice. For example, when the electronic device is a smart speaker, the user can say "Hey Celia" to wake it up and request music; the smart speaker receives the user's voice, analyzes and processes it, and then plays music. Summary of the Invention

[0003] Electronic devices with multiple microphones can receive user voice commands and enable voice control. However, such devices are vulnerable to light wave attacks (e.g., laser attacks). For example, when a pre-programmed laser beam shines from outside onto a microphone of a smart speaker inside a house, the microphone may demodulate the laser signal into an electrical signal. Outdoors, by adjusting the frequency and amplitude of the laser signal, the smart speaker could be activated, potentially controlling other smart devices (e.g., unlocking a smart door lock), posing a significant security risk to the user. Therefore, providing a method to prevent light wave attacks on electronic devices with multiple microphones, reducing the security risks caused by attacks from various light waves, including lasers, has become a necessity.

[0004] To achieve the above-mentioned technical objectives, this application provides a method, electronic device, computer-readable storage medium, and computer program product for preventing light wave attacks.

[0005] Firstly, a method for preventing light wave attacks is provided, applied to an electronic device comprising n microphones. The method includes: acquiring an audio signal containing a wake-up word through u microphones among the n microphones; if the energy value corresponding to at least one of the u microphones deviates from a reference energy value of the n microphones within a preset range, the electronic device outputs an alarm message and is not woken up by the audio signal; if the energy value corresponding to each of the u microphones deviates from the reference energy value of the n microphones within a preset range, the electronic device is woken up by the audio signal; wherein n is a positive integer greater than or equal to 1; and u is a positive integer greater than or equal to 1 and less than n.

[0006] According to the first aspect, after the energy value corresponding to at least one of the u microphones deviates from the reference energy value of the n microphones outside a preset range, and before the electronic device outputs an alarm message and is not woken up by the audio signal; the method further includes: the reference energy value being less than or equal to a preset upper limit value. Thus, after the u microphones on the electronic device collect an audio signal containing a wake-up word, the method can determine whether to wake up the electronic device based on a comparison between the energy value corresponding to at least one of the u microphones and the reference energy value of the n microphones, reducing the security risks caused by attacks on electronic devices from various light waves, including lasers.

[0007] According to the first aspect, or any implementation of the first aspect above, before acquiring an audio signal containing a wake-up word through u microphones out of n microphones, the method further includes: determining that k microphones out of the n microphones are malfunctioning; before the electronic device outputs an alarm message and is not woken up by the audio signal, the method further includes: at least one microphone among the u microphones is not located among the k microphones, nor does it overlap with the k microphones; where k is a positive integer greater than or equal to 0 and less than n. In this way, it is determined that the pickup capability of at least one microphone among the u microphones is normal, thus concluding that the electronic device is at risk of being attacked by light waves.

[0008] According to the first aspect, or any implementation of the first aspect above, the electronic device further includes an audio module and a processor; n microphones are connected to the processor through the audio module; the audio module is used to sample and convert the analog audio signal transmitted from any one of the n microphones into an analog-to-digital signal, and transmit the converted digital signal to the processor.

[0009] The energy value for each of the u pickups is calculated as follows:

[0010] The number of sampling points corresponding to each of the u microphones is M, and the amplitude value of the q-th sampling point among the M sampling points corresponding to the p-th microphone is V. p (q), V p (q) The digital signal value after analog-to-digital conversion is D p (q); According to formula (1), the energy value D corresponding to the p-th pickup among u pickups is calculated. p ;

[0011]

[0012] Where p is a positive integer greater than or equal to 1 and less than or equal to u; q is a positive integer greater than or equal to 1 and less than or equal to M. In this way, the energy value corresponding to each of the u pickups can be obtained.

[0013] According to the first aspect, or any implementation of the first aspect above, the baseline energy value is calculated as follows:

[0014] The number of sampling points corresponding to each of the (nu) pickups is M. The (nu) pickups are the pickups other than u among the n pickups. The amplitude value of the q-th sampling point among the M sampling points corresponding to the r-th pickup in the (nu) pickups is V. r (q), V r (q) The digital signal value after analog-to-digital conversion is D r (q); According to formula (2), the energy value D corresponding to the r-th pickup among (nu) pickups is calculated. r ;

[0015]

[0016] Where r is a positive integer greater than or equal to 1 and less than or equal to (nu);

[0017] According to formula (3), the maximum energy value D of u energy values ​​is calculated. max1 ;

[0018] D max1 =max 1≤p≤u D p Equation (3)

[0019] According to formula (4), the maximum energy value D of (nu) energy values ​​is calculated. max2 ;

[0020] D max2 =max 1≤r≤(n-u) D r Equation (4)

[0021] According to D max1 and D max2 D was calculated max1 and D max2 The maximum value in the range is the reference energy value D. max0 .

[0022] According to the first aspect, or any implementation of the first aspect above, the deviation between the energy value of at least one of the u pickups and the reference energy value of the n pickups is calculated as follows:

[0023]

[0024] Among them, b p For deviation

[0025] According to the first aspect, or any implementation of the first aspect above, the deviation between the energy value of at least one of the u pickups and the reference energy value of the n pickups is calculated as follows:

[0026] b p =D max0 -D p Equation (6)

[0027] Among them, b p This is a deviation.

[0028] According to the first aspect, or any implementation of the first aspect above, the preset range includes one of the following: [0,b0], [0,b0), (0,b0), (0,b0]; where b0 is a preset second threshold.

[0029] According to the first aspect, or any implementation of the first aspect above, the malfunction of k out of n microphones is obtained as follows: the audio module receives another analog audio signal collected by the n microphones, and after sampling and analog-to-digital conversion, obtains another digital signal after analog-to-digital conversion; based on the other digital signal after analog-to-digital conversion corresponding to each of the n microphones, the first energy value corresponding to each microphone is calculated, thereby obtaining n first energy values; based on the n first energy values, the maximum value D of the n first energy values ​​is obtained. max According to D max And for each of the n first energy values, calculate D. max Another deviation between each first energy value and the second deviation is calculated, resulting in n additional deviations. Based on these n additional deviations, k out of the n microphones are found to be faulty. Then, the energy value of the audio signal collected by each microphone is compared with the largest of these energy values ​​to identify the faulty microphone.

[0030] According to the first aspect, or any implementation of the first aspect above, the n first energy values ​​are obtained as follows: The first energy value D corresponding to the i-th pickup among the n pickups is calculated according to formula (1). i ;

[0031]

[0032] Among them, D i Let M be the first energy value corresponding to the i-th pickup out of n pickups, and M be the total number of sampling points for the i-th pickup; D i (j) is the amplitude value of the sampling point corresponding to the i-th microphone among the n microphones, and the digital signal value after analog-to-digital conversion.

[0033] Based on the first aspect, or any implementation of the first aspect above, the maximum value D of the n first energy values ​​is obtained from the n first energy values. max It was obtained in the following way:

[0034] According to formula (2), the maximum value D of the n first energy values ​​is obtained. max ;

[0035] D max =max 1≤i＜≤n D i Formula (2).

[0036] According to the first aspect, or any implementation of the first aspect above, according to D max And for each of the n first energy values, calculate D. max Another deviation from each first energy value is obtained as follows:

[0037]

[0038] Among them, a i This represents the i-th deviation corresponding to the i-th pickup among n pickups.

[0039] According to the first aspect, or any implementation of the first aspect above, according to D max And for each of the n first energy values, calculate D. max Another deviation from each first energy value is obtained as follows:

[0040] a i =D max -D i Formula (4)

[0041] Among them, a i This represents the i-th deviation corresponding to the i-th pickup among n pickups.

[0042] According to the first aspect, or any implementation of the first aspect above, the determination that k microphones are abnormal among the n microphones is obtained by the following method: among the n additional deviations, the number of additional deviations greater than the preset first threshold a0 is k.

[0043] According to the first aspect, or any implementation of the first aspect above, the n pickups include all or part of at least one of the following: microphones, microphone arrays.

[0044] In a second aspect, an electronic device is provided, comprising n microphones, an audio module, a memory, and a processor; the n microphones are connected to the processor via the audio module; the audio module is used to sample and convert analog audio signals transmitted from any one of the n microphones into digital signals, and transmit the converted digital signals to the processor; the memory stores a computer program; when the computer program is executed by the processor, the electronic device performs the first aspect and any implementation thereof.

[0045] The second aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the second aspect and any implementation thereof can be found in the first aspect and any implementation thereof, and will not be repeated here.

[0046] Thirdly, a computer-readable storage medium is provided, the computer-readable storage medium including a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in the second aspect and any implementation thereof.

[0047] The third aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects of the third aspect and any implementation thereof can be found in the technical effects of the second aspect and any implementation thereof, as described above, and will not be repeated here.

[0048] Fourthly, a computer program product is provided that, when run on an electronic device, causes the electronic device to perform the method as described in the second aspect and any implementation thereof.

[0049] The fourth aspect and any implementation thereof correspond to the second aspect and any implementation thereof, respectively. The technical effects of the fourth aspect and any implementation thereof can be found in the technical effects of the second aspect and any implementation thereof, as described above, and will not be repeated here. Attached Figure Description

[0050] Figure 1 This is a schematic diagram illustrating an application scenario provided in the embodiments of this application;

[0051] Figure 2 A schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application;

[0052] Figure 3 A flowchart illustrating the method for preventing light wave attacks provided in this application embodiment;

[0053] Figure 4 A flowchart illustrating the initial detection step of the microphone in the method for preventing light wave attacks provided in this application embodiment;

[0054] Figure 5 A schematic diagram illustrating the principle of the audio module provided in this application for sampling the analog audio signal acquired by the i-th microphone;

[0055] Figure 6 A flowchart illustrating the detection and response steps for an audio signal containing a wake word in a method for preventing light wave attacks provided in this application embodiment;

[0056] Figure 7 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0057] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two). The term “and / or” is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0058] References to "one embodiment" or "some embodiments" as used in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes both direct and indirect connections, unless otherwise stated.

[0059] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0060] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0061] For illustrative purposes, the following example uses electronic device 200 as a smart speaker, and the multiple pickups in electronic device 200 as multiple microphones. Figure 1 This application aims to clarify its application scenarios. It should be noted that the electronic device in this application is not limited to a smart speaker. Other electronic devices with multiple microphones are also within the scope of this application. Similarly, the microphone in this application is not limited to a microphone.

[0062] For example, Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. For example... Figure 1 As shown, the electronic device 200 is equipped with four microphones—microphones 111, 112, 113, and 114. It should be noted that the four microphones are merely illustrative examples and are not intended to limit the number of microphones. The number of microphones can be any other number, as long as it satisfies the requirement of multiple (two or more). Optionally, the electronic device 200 can be equipped with multiple microphone arrays, each of which can include multiple microphones. For example, microphone arrays 111, 112, 113, and 114. Optionally, the electronic device 200 can be equipped with a combination of microphone arrays and microphones. For example, microphone array 111, microphone 112, microphone array 113, and microphone 114. The order of the microphone arrays and microphones described above is merely illustrative. Other orders are also within the scope of this application. The pickup entrance of the microphones or microphone arrays described above can be located on the upper surface of the electronic device 200. Furthermore, the pickup entrance can also be located on other surfaces of the electronic device 200, which is not limited here.

[0063] For example, an attacker can use a light beam 121 generated by a light wave irradiation device 12 (e.g., a laser pointer) to irradiate one or more microphones on an electronic device 200 (e.g., irradiating the pickup inlet of microphone 111, thus irradiating microphone 111). The attacker can then modulate the light beam 121, such as amplitude modulation, frequency modulation, phase modulation, etc. (or the signal can be pre-set, and the light beam 121 can be further adjusted as needed, or no further adjustment is required). This allows the attacker to simulate the sound wave corresponding to a wake-up word using light waves, causing the microphone on the electronic device 200 (e.g., microphone 111) to recognize the light wave signal as the voice signal corresponding to the wake-up word, thereby waking up the electronic device 200 and controlling it, as well as controlling other smart devices through the electronic device 200 (e.g., controlling the unlocking of a smart door lock), thus posing a significant security risk to the user.

[0064] To mitigate security risks posed by light wave attacks, such as laser attacks, and to ensure the security of electronic devices and improve user security, this application provides a method and electronic device for preventing light wave attacks. The method for preventing light wave attacks provided in this application is applied to electronic devices with multiple microphones. The electronic devices can be smart home devices such as smart speakers, smart TVs, smart air conditioners, smart refrigerators, smart lights, smart doors, smart locks, and smart curtains; wearable electronic devices such as smart glasses, smartwatches, and smart bracelets; smartphones, tablets, laptops, personal digital assistants (PDAs), in-vehicle devices, virtual reality devices, and augmented reality devices, etc., with voice interaction capabilities. This application does not impose any limitations on these.

[0065] For example, Figure 2 A schematic diagram of the hardware structure of an electronic device according to an embodiment of this application is shown. Figure 2 As shown, the electronic device may include a processor 210, a memory 220, an audio module 230, multiple microphones 240, a speaker 250, and a communication module 260.

[0066] The processor 210 can be a general-purpose processor or a dedicated processor. For example, the processor 210 may include a central processing unit (CPU) and / or a baseband processor. The baseband processor can be used to process communication data, while the CPU can be used to implement corresponding control and processing functions, execute software programs, and process data from the software programs. For example, the processor 210 can determine the microphone's pickup capability based on the audio signal collected by the microphone 240, or determine whether it is currently using light wave wake-up based on the audio signal collected by the microphone 240 when the electronic device meets the wake-up conditions. For example, the processor 210 performs automatic speech recognition (ASR), natural language understanding (NLU), dialogue management (DM), natural language generation (NLG), and text-to-speech (TTS) on the audio signal collected by the microphone 240.

[0067] Exemplarily, processor 210 may include one or more processing units. For example, processor 210 may include one or more of the following: application processor (AP), modem, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural-network processing unit (NPU). In some embodiments, the electronic device may include one or more processors 210. Different processing units may be independent devices or integrated into one or more processors. Exemplarily, processor 210 may include an application processor (AP) and a digital signal processor (DSP). Data output from audio module 230 may first be sent to the DSP for processing and then to the AP for processing. Thus, data preprocessing, such as noise reduction, by the DSP improves processing speed.

[0068] The memory 220 may store a program, which can be executed by the processor 210 to perform the methods provided in this application. The memory 220 may also store data. The processor 210 can read the data stored in the memory 220 (e.g., audio data, pre-marked identifiers of microphones with abnormal pickup capabilities, etc.). The memory 220 and the processor 210 may be configured separately. Alternatively, the memory 220 may be integrated into the processor 210.

[0069] A microphone 240, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. Electronic devices include multiple microphones 240. Microphones 240 can collect sound from the environment in which the electronic device is located. Some (e.g., one or more) of the multiple microphones 240 may be attacked by light waves. Optionally, each microphone can be pre-numbered to distinguish it from others. Furthermore, the sampling frequency of each microphone can be pre-calibrated. Optionally, some of the multiple microphones 240 are microphones, and others are microphone arrays. Optionally, all of the multiple microphones 240 can be microphones or microphone arrays. Optionally, the sound pickup inlets of the multiple microphones 240 can be located on the upper surface, lower surface, or side surface of the electronic device. Figure 1 111, 112, 113, and 114 are examples of multiple microphones 240. Optionally, some of the microphones 24 have pickup entrances located on the upper, lower, or side surface of the electronic device, while other microphone entrances are located on other surfaces of the electronic device.

[0070] Audio module 230 samples analog sound signals acquired by multiple microphones 240 according to a preset sampling frequency (which can be set by the processor, at the factory, or by default), converts the sampled analog sound signals into digital audio signals, and then inputs the digital audio signals to the processor. Optionally, audio module 230 can also convert digital audio signals into analog sound signals and output them to speaker 250. Audio module 230 is also used for encoding and decoding audio signals. In some examples, audio module 230 can be integrated into processor 21, or some functions of audio module 230 can be integrated into processor 210. The number of audio modules can be one or more. Audio module 230 may include an analog-to-digital converter (ADC). Optionally, the electronic device includes multiple audio modules 230. One microphone from the multiple microphones 240 is connected to one audio module 230, meaning the number of audio modules is the same as the number of microphones. Optionally, the number of audio modules may be less than the number of microphones. For example, let P be the number of audio modules and Q be the number of microphones; P < Q, where both P and Q are positive integers greater than or equal to 1. In this case, R microphones out of the Q microphones can correspond to one audio module; R is a positive integer less than or equal to Q. When R microphones correspond to one audio module, the audio module can process the sound collected by each of the R microphones in a pre-defined order. The sound collected by the microphones can be transmitted to the corresponding audio module for processing; the processed sound is then transmitted to the processor.

[0071] For example, assume there are two microphones and one audio module, with the two microphones connected to the audio module. For example, the audio module has a sampling frequency of 16 kilohertz (kHz), i.e., a sampling period of 0.0625 milliseconds (ms). For ease of explanation, assume the two microphones are microphone A and microphone B. The analog sound signal acquired by microphone A is signal A1, and the analog sound signal acquired by microphone B is signal B1. Microphone A transmits signal A1, and microphone B transmits signal B1, to the audio module. The audio module first samples signal A1 for one cycle, and then performs an analog-to-digital (A / D) conversion after that cycle. The converted signal is denoted as digital audio signal A2. Next, the audio module samples signal B1 for one cycle, and then performs an A / D conversion after that cycle. The converted signal is denoted as digital audio signal B2. Then, the audio module samples signal A1 for the next cycle, and then performs an A / D conversion after that cycle. The converted signal is combined with the previous digital audio signal A2 to form a new digital audio signal A2. This process is repeated until all analog audio signals have been sampled and converted. In this way, A2 and B2 are continuously incremented until the analog audio signals corresponding to each microphone have been converted. Finally, A2 and B2 are transmitted to the processor.

[0072] Optionally, the analog audio signal corresponding to each microphone can be transmitted to the processor after reaching a certain proportion or quantity of sampling and A / D conversion (e.g., completing one cycle of sampling and A / D conversion). Subsequently, the proportion or quantity of each microphone's analog audio signal is re-accumulated, and transmitted to the processor again after reaching a certain proportion or quantity (e.g., completing another cycle of sampling and A / D conversion); ...; this cycle continues until all analog audio signals corresponding to each microphone have been converted and transmitted. The processor can perform subsequent processing based on the two received digital audio signals corresponding to one cycle. Alternatively, the processor can perform subsequent processing on the two received digital audio signals that have completed all conversions.

[0073] In some embodiments, after the audio module completes one cycle of sampling of sound signal A1, it then samples sound signal B1 for one cycle. Simultaneously with sampling sound signal B1 for one cycle, the audio module also performs an analog-to-digital (AD) conversion on the sampling result of sound signal A1 from the previous cycle. That is, the sampling and AD conversion functions of the audio module are independent of each other and executed synchronously. Those skilled in the art will understand that the sampling, AD conversion, and even the converted signal transmission functions of the audio module can be considered as functions executed by three different sub-modules, which execute synchronously and independently.

[0074] For example, the number of microphones is the same as the number of audio modules. Each microphone is connected to one audio module. Each microphone transmits the acquired sound signal to the corresponding audio module. The audio module samples the sound signal (e.g., sampling in one cycle, then sampling for the next cycle, and so on), performs an A / D conversion after sampling, and transmits the converted signal to the processor.

[0075] For example, an audio module may include multiple ADCs. For instance, an audio module may include S ADCs; thus, S microphones can be connected to an audio module; S is a positive integer greater than 1. In essence, it is still one microphone connected to one ADC. For details regarding sampling, AD conversion, etc., please refer to the above content.

[0076] Speaker 25, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. For example, when the electronic device is a smart speaker, the smart speaker can play music through speaker 25.

[0077] The communication module 26 may include at least one of a mobile communication module and a wireless communication module. When the communication module 26 includes a mobile communication module, it can provide wireless communication solutions for electronic devices, including 2G / 3G / 4G / 5G. Examples include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), and New Radio (NR).

[0078] The communication module 26 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The communication module 26 can receive electromagnetic waves using at least two antennas, including an antenna, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem for demodulation. The communication module 26 can also amplify the signal modulated by the modem and radiate it as electromagnetic waves via the antennas. In some examples, at least some functional modules of the communication module 26 may be housed in the processor 21. In some examples, at least some functional modules of the communication module 26 and at least some modules of the processor 21 may be housed in the same device. When the communication module 26 includes a wireless communication module, it can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The communication module 26 can be one or more devices integrating at least one communication processing module. The communication module 26 receives electromagnetic waves via an antenna, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to the processor 21. The communication module 26 can also receive signals to be transmitted from the processor 21, perform frequency modulation and amplification, and then convert them into electromagnetic waves for radiation via the antenna.

[0079] Optionally, the electronic device may include a display screen. For example, the display screen may show the interface of an application, the display window of an application (e.g., a music card for music played by the electronic device), etc. For example, the display screen may also display a notification message reminding the user that they are currently under laser attack.

[0080] Alternatively, electronic devices can communicate with external devices through various interfaces. For example, they can communicate with external devices through Universal Serial Bus (USB) interfaces, Ethernet interfaces, FireWire interfaces, etc.

[0081] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0082] It should be noted that the electronic device of this application not only includes multiple microphones, but also has voice control function.

[0083] The following is combined Figure 3 The technical solution provided in this application will be described in detail below. For example, Figure 3 A flowchart illustrating a method for preventing light wave attacks provided in an embodiment of this application is shown. Figure 3 As shown, the method may include the following steps:

[0084] S1. Initial testing steps for the microphone.

[0085] Specifically, the initial detection step for the microphone is used to initially detect whether the microphone's pickup capability is abnormal. In this application, the pickup capability of multiple microphones on the electronic device can be detected in advance to identify any microphones with abnormal pickup capability. For example, the electronic device can automatically perform an initial detection once at a predetermined time each day (week, month, quarter, half-year, etc.) (e.g., 7:00 AM every day). If no abnormality is found in the initial detection, no further detection is performed on the same day (week, month, quarter, half-year, etc.). In this way, if a user encounters a situation such as a laser attack or light wave attack in the afternoon after waking up and going to work in the morning, the electronic device (e.g., a smart speaker) does not need to execute S1.

[0086] As one possible implementation, the electronic device can continuously or periodically execute S1 to determine whether the pickup capabilities of multiple microphones on the electronic device are abnormal. Specifically, when the electronic device detects that one or more microphones have abnormal pickup capabilities, it can mark those microphones as abnormal. For example, if the first microphone on the electronic device has abnormal pickup capabilities, the electronic device can mark that microphone as abnormal.

[0087] As another possible implementation, the microphones on the electronic device can continuously or periodically receive audio signals from the environment. The electronic device can then detect the pickup capability of each microphone based on the audio signals received by each microphone.

[0088] It should be noted that S1 is not a necessary step in this method. That is, S1 is not executed every time the method is executed. Alternatively, S1 can be executed every time the method is executed.

[0089] After detecting the pickup capability of the microphone, S2 can be executed.

[0090] S2, the detection and response steps for the acquired audio signal containing the wake word.

[0091] Specifically, the detection and response steps for acquiring an audio signal containing a wake-up word are used to detect whether the acquired audio signal containing the wake-up word is caused by a light wave attack when the electronic device is woken up, and to respond accordingly. For example, if the electronic device detects that the audio signal containing the wake-up word is caused by a light wave attack, it can output an alarm message and not be woken up by the audio signal; if the electronic device detects that the audio signal containing the wake-up word is not caused by a light wave attack, it can be woken up by the audio signal.

[0092] In this application, when some (e.g., one) or all of the audio signals collected by multiple microphones on an electronic device meet the wake-up condition, the electronic device executes S2 to detect whether the currently collected audio signal containing the wake-up word is caused by a light wave (e.g., laser) attack.

[0093] The following is combined Figure 4 The initial detection steps for the microphone are described in detail in the method for preventing light wave attacks provided in this application. For example, Figure 4 This is a flowchart illustrating the initial detection step of the microphone in the method for preventing light wave attacks provided in this application embodiment. Figure 4 In the illustrated process, the electronic device includes n microphones, the audio module of the electronic device has a sampling period of T, a sampling frequency of f (f = 1 / T), m sampling points in each sampling period, N sampling periods, and a total of M sampling points. The amplitude value of the j-th sampling point among the M sampling points corresponding to the i-th microphone is represented by f. i (j) indicates that f i (j) The signal after A / D conversion is used with D i (j) indicates.

[0094] Where n, m, N, and M are all positive integers greater than or equal to 1, i is less than or equal to n, and j is less than or equal to M. For example... Figure 4 As shown, the initial detection steps for the microphone may include:

[0095] S11. The audio module receives analog audio signals collected by n microphones, performs sampling and AD conversion, and sends the digital audio signal after AD conversion to the processor.

[0096] Each of the n microphones collects audio signals from the environment and then sends these signals to the audio module. The audio module samples and performs analog-to-digital (A / D) conversion, then transmits the converted digital audio signal to the processor. The sampling and A / D conversion of the analog audio signals collected by the n microphones by the audio module can be performed using the sampling and A / D conversion methods described earlier.

[0097] Option 1: The audio module samples the audio signal from the first pickup in the first sampling period, then the second pickup, and so on, until the nth pickup is sampled. This process is repeated in the second sampling period until no more audio signals are received from the pickups. Similarly, the AD conversion is performed synchronously and independently in the same order. In this way, each of the n pickups receives M sampling points, resulting in a total of n*M sampling points. The amplitude value of the j-th sampling point among the M sampling points corresponding to the i-th pickup is f. i (j), f i (j) After AD conversion, it becomes D i (j). Thus, the digital signal D, obtained by A / D conversion, is derived from the amplitude value of the sampling point corresponding to the i-th microphone among the n microphones over N sampling periods. i (j).

[0098] For example, Figure 5 The waveform of the analog audio signal received by the audio module from the i-th microphone is shown. For ease of explanation, Figure 5 The waveform of the analog audio signal received by the audio module from the i-th microphone is shown only for 3 sampling periods.

[0099] For example, see [link to relevant documentation]. Figure 5 Within the 0-T period, samples can be taken at times t1, t2, and t3, resulting in three sampling points. The sampling point at time t1 is designated j1, at time t2 j2, and at time t3 j3. The amplitude value corresponding to sampling point j1 is f. i (j1), the amplitude value corresponding to sampling point j2 is f i (j2), the amplitude value corresponding to sampling point j3 is f i (j3). The amplitude value is f i (j1) becomes D after AD conversion. i (j1), amplitude value f i (j2) becomes D after AD conversion.i (j2), amplitude value f i (j3) becomes D after AD conversion. i (j3).

[0100] Option 2: The audio module samples the audio signal from the first pickup only during the first sampling period, then the second pickup, and so on, until the nth pickup. After that, it stops sampling the audio signal from each of the n pickups during the second sampling period. Similarly, the AD conversion is also performed synchronously and independently during the first sampling period, following the above sequence. In other words, only the first sampling period is selected for sampling and AD conversion of the audio signal from each of the n pickups. This way, only the amplitude value of the sampling point corresponding to each of the n pickups is obtained after AD conversion, resulting in the digital signal value D. i (j). The sampling process can be referred to the previous description, and will not be repeated here. However, in the method described in that section, M = m.

[0101] S12. Based on the digital audio signal after AD conversion corresponding to each of the n microphones, calculate the first energy value corresponding to each microphone, thereby obtaining n first energy values.

[0102] The digital audio signal after AD conversion for each of the n microphones, including the aforementioned D... i (j) and information indicating that the sampling point is the i-th sampling point. Based on the above digital audio signal, the processor can calculate the first energy value corresponding to each microphone using the following formula (1), and then obtain n first energy values.

[0103]

[0104] Among them, D i Let M be the first energy value corresponding to the i-th pickup out of n pickups, and D be the total number of sampling points. i (j) is the digital signal value of the sampling point corresponding to the i-th microphone among the n microphones after AD conversion. In the first method described above, M in formula (1) is greater than or equal to m. In the second method described above, M = m in formula (1). In the second method described above, S12 is essentially a calculation of the digital audio signal corresponding to each of the n microphones in the same sampling period (e.g., the first sampling period).

[0105] For example, such as Figure 5The calculation process of the energy value is illustrated using an example with 3 sampling points within the 0-T period. The 3 sampling points are j1, j2, and j3, and the amplitude value corresponding to sampling point j1 is f. i (j1), the amplitude value corresponding to sampling point j2 is f i (j2), the amplitude value corresponding to sampling point j3 is f i (j3). The amplitude value is f i (j1) becomes D after AD conversion. i (j1), amplitude value f i (j2) becomes D after AD conversion. i (j2), amplitude value f i (j3) becomes D after AD conversion. i (j3). D i (j1), D i (j2) and D i Substituting (j3) into formula (1), we get:

[0106]

[0107] S13. Based on the n first energy values, obtain the maximum value D of the n first energy values. max .

[0108] After obtaining n first energy values, the largest first energy value D can be obtained according to the following formula (2). max Alternatively, the largest first energy value D can be obtained by sorting, for example, by sorting the n first energy values ​​from largest to smallest or smallest to largest. max .

[0109] D max =max 1≤i＜≤n D i Formula (2)

[0110] S14, according to D max For each of the n first energy values, evaluate D. max The deviation between the first energy value and each of the n first energy values.

[0111] For example, according to D max For each of the n first energy values, n deviations are obtained. Specifically, the i-th deviation 'a' corresponds to the i-th pickup. i It can be calculated using the following formula (3).

[0112]

[0113] Among them, D max D iPlease refer to the previous description; it will not be repeated here.

[0114] Alternatively, the i-th deviation a corresponding to the i-th pickup i It can be calculated using the following formula (4).

[0115] a i =D max -D i Formula (4)

[0116] It should be noted that formulas (3) and (4) above are only illustrative examples, and the relevant information regarding the i-th deviation a is not provided in the original text. i The calculation formulas include, but are not limited to, formulas (3) and (4) mentioned above. Any formula capable of evaluating D... max With D i The formulas relating the deviations between them are all within the scope of this application.

[0117] Alternatively, to improve processing efficiency, D can be omitted from calculation. max The corresponding deviation. According to D max and except D max For each of the (n-1) first energy values ​​other than the given first energy values, (n-1) deviations are obtained. The specific deviations can be adjusted according to the actual situation and are not limited here. This allows for subsequent calculations based on the (n-1) deviations.

[0118] S15. Based on n deviations, obtain k microphones with abnormal pickup capabilities, where k < n.

[0119] After obtaining n deviations, each deviation is compared with a first threshold a0, thus identifying k microphones out of the n microphones with abnormal pickup capabilities; where k < n, and k is a positive integer greater than or equal to 1. It is understandable that when the pickup capabilities of all microphones are normal, the analog audio signals collected by each microphone are relatively similar, and therefore the corresponding first energy values ​​of each microphone are also relatively similar. Thus, the deviation corresponding to each microphone is less than or equal to the first threshold a0. That is, when the deviation corresponding to the i-th microphone is greater than the first threshold a0, it can be determined that the i-th microphone has an abnormal pickup capability. At this time, the pickup capability of the i-th microphone can be marked as abnormal, or the i-th microphone can be marked as an abnormal microphone. For example, the first threshold a0 can be 0.5. Of course, the first threshold a0 can also be other values, which are not limited here.

[0120] Optionally, a first energy value upper limit D corresponding to the i-th pickup can be preset. thi When D i Greater than D thi This indicates that D iThere is an error; in this case, D can be recalculated based on the sampling results. i And the recalculated D i With D thi Compare; or, you can reacquire the analog audio signals from each pickup, resample, recalculate, etc.; until D i Less than or equal to D thi .

[0121] It should be noted that different upper limits for the first energy value can be preset for different pickups (for example, the upper limit D of the first energy value corresponding to the i-th pickup). thi The upper limit of the first energy value D corresponding to the j-th pickup thj (Not equal), or the same first energy value upper limit can be preset for different pickups (for example, the first energy value upper limit D corresponding to the i-th pickup). thi The upper limit of the first energy value D corresponding to the j-th pickup thj Equal, both equal to D th Optionally, for each of the n pickups, the same initial energy limit can be preset to D. th Optionally, the same first energy value upper limit can be preset for some of the n microphones; different first energy value upper limits can be preset for the other n microphones.

[0122] For example, the specific steps of S15 can be as follows: Figure 4 As shown in (B). Specifically, S15 may include:

[0123] S151. Determine whether the i-th deviation corresponding to the i-th pickup is greater than the first threshold a0.

[0124] By comparing the i-th deviation with the first threshold a0, the magnitude between the two can be determined. For example, if the first threshold a0 is 10, and the i-th deviation is 5, then the i-th deviation is less than the first threshold a0; if the i-th deviation is 15, then the i-th deviation is greater than the first threshold a0.

[0125] When the i-th deviation is less than or equal to the first threshold a0, it indicates that the first energy value D corresponding to the i-th pickup is... i The maximum value D of the n first energy values max The deviation between them is small, which indicates that the pickup capability of the i-th pickup is normal, that is, S152 is executed.

[0126] When the i-th deviation is greater than the first threshold a0, it indicates that the first energy value D corresponding to the i-th pickup is... i The maximum value D of the n first energy values maxThe deviation between them is small, indicating that the pickup capability of the i-th pickup is abnormal. At this time, S154 can be executed. Optionally, considering that when D max It is possible that some kind of error will result in the obtained D. max There is a problem; to ensure accuracy, you can execute S153 first.

[0127] S152. The pickup capability of the i-th pickup is normal; i = i + 1, and determine whether the updated i is less than or equal to n.

[0128] At this point, there is no need to mark the i-th microphone as an abnormal microphone, or mark the pickup capability of the i-th microphone as abnormal. Then, update i by i = i + 1. Determine if the updated i is less than or equal to n. If it is less than or equal to n, execute S151. If it is greater than n, end the process, and record which microphones have abnormal pickup capabilities, or which microphones are abnormal. Optionally, the abnormal pickup capability information can also be displayed on a screen, and / or played through a speaker.

[0129] S153, Determine D max Is it greater than the preset first energy value limit D? th .

[0130] D max With the pre-set first energy value limit D th Compare and determine the magnitude of the two. For example, when the first energy value upper limit D... th If D is 20, then... max If it is 15, then D max Less than the upper limit of pickup D th If D max If the value is 25, then D max Greater than the first energy value limit D th .

[0131] Where, when D max Less than or equal to the first energy value upper limit D th When, it indicates D max Normal. At this point, it can be determined that the pickup capability of the i-th pickup is abnormal, i.e., execute S154.

[0132] When D max Greater than the first energy value limit D th When, it indicates D max An error occurred; at this point, execute S155 to exit the process, or recalculate D. max Then rerun the process.

[0133] S154. The pickup capability of the i-th pickup is abnormal; i = i + 1, and determine whether the updated i is less than or equal to n.

[0134] At this point, the i-th microphone is marked as an abnormal microphone, or its pickup capability is marked as abnormal. Then, i is updated by i = i + 1. It is then determined whether the updated i is less than or equal to n. If it is less than or equal to n, step S151 is executed. If it is greater than n, the process ends, and the records of which microphones have abnormal pickup capabilities, or which microphones are abnormal, are recorded. Optionally, the abnormal pickup capability information can also be displayed on a screen, and / or played back through a speaker.

[0135] S155. Exit the process, or recalculate D. max Then rerun the process.

[0136] Optionally, to improve the accuracy of detecting abnormal pickup capability of the i-th microphone and reduce detection errors caused by the randomness of relying solely on a single detection, other methods can be used to further confirm the detection when an abnormal pickup capability of the i-th microphone is detected. For example, when an abnormal pickup capability of the i-th microphone is detected, this moment is taken as time t0. If an abnormal pickup capability of the i-th microphone is detected again or multiple times within a period of time (e.g., 30 minutes) starting from t0, then the pickup capability of the i-th microphone is determined to be abnormal. If the pickup capability of the i-th microphone is detected again or multiple times within a period of time (e.g., 30 minutes) starting from t0, then the pickup capability of the i-th microphone is determined to be normal.

[0137] The following is combined Figure 6 This application provides a detailed description of the detection and response steps for acquiring audio signals containing wake words in the method for preventing light wave attacks. For example, Figure 6 This is a flowchart illustrating the detection and response steps for an audio signal containing a wake-up word in a method for preventing light wave attacks provided in an embodiment of this application. Figure 6 In the illustrated process, the electronic device includes n microphones. u of these n microphones collect the audio signal containing the wake-up word. The sampling period of the electronic device's audio module is T, the sampling frequency is f (f = 1 / T), the number of sampling points in each sampling period is m, the number of sampling periods is N, and the total number of sampling points is M. The amplitude value of the q-th sampling point among the M sampling points corresponding to the p-th microphone in the u microphones is represented by f. p (q) means that f p (q) The signal after A / D conversion is used by D p(q) represents the amplitude value of the q-th sample point among the M sample points corresponding to the r-th pickup in (nu) pickups, expressed as V. r (q) indicates that V r (q) The signal after A / D conversion is used by D r (q) represents.

[0138] Among them, (nu) pickups are the pickups other than u pickups out of n pickups, n, u, m, N, and M are all positive integers greater than or equal to 1, u is less than n, p is less than or equal to u, r is less than or equal to (nu), and q is less than or equal to M.

[0139] like Figure 6 As shown, the detection and response steps for acquiring audio signals containing wake words may include:

[0140] S201. The audio module receives analog audio signals collected by u microphones out of n microphones, performs sampling and AD conversion, and sends the digital audio signal after AD conversion to the processor, where 1≤u<n.

[0141] The u microphones out of n microphones collect audio signals from the environment and then send these signals to the audio module. The audio module samples and performs analog-to-digital (A / D) conversion on these signals, before transmitting the converted digital audio signal to the processor. The sampling and A / D conversion of the analog audio signals collected by the u microphones by the audio module can be performed using the sampling and A / D conversion methods described earlier.

[0142] Option 1: The audio module samples the audio signal from the first pickup in the first sampling period, then the second pickup, and so on, until the u-th pickup is sampled. Then, in the second sampling period, the same process is repeated until no more audio signals are received from the pickups. Similarly, the AD conversion is performed synchronously and independently in the same order. In this way, each of the u pickups receives a total of M sampling points, resulting in a total of u*M sampling points. The amplitude value of the q-th sampling point among the M sampling points corresponding to the p-th pickup is f. p (q), f p (q) becomes D after A / D conversion. p (q). Thus, the digital signal D, obtained by A / D conversion, is derived from the amplitude value of the sampling point corresponding to the p-th pickup among the n pickups over N sampling periods. p (q).

[0143] Option 2: The audio module samples the audio signal from the first pickup only during the first sampling period, then the second pickup, and so on, until the u-th pickup. After that, it stops sampling the audio signal from each of the u pickups during the second sampling period. Similarly, the AD conversion is also performed synchronously and independently during the first sampling period, following the above sequence. In other words, only the first sampling period is selected for sampling and AD conversion of the audio signal from each of the u pickups. This way, only the amplitude value of the sampling point corresponding to each of the u pickups is obtained after AD conversion, resulting in the digital signal value D. p (q). The sampling process can be referred to the previous description, and will not be repeated here. However, in the method described in that section, M = m.

[0144] Understandably, the audio module can also receive analog audio signals from (nu) out of n microphones, sample and perform A / D conversion, and then send the converted digital audio signal to the processor. The sampling and A / D conversion of the analog audio signals from (nu) microphones by the audio module can be referred to the description above of the sampling and A / D conversion of the analog audio signals from u microphones, and will not be repeated here.

[0145] S202. Determine whether the digital audio signal after AD conversion for each of the u microphones contains a wake-up word.

[0146] The processor receives the digital audio signals converted from analog to digital (A / D) signals from each of the u microphones. Then, the processor analyzes and processes each of the u digital audio signals, performing actions such as Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU), to determine if at least one of the u digital audio signals contains a wake-up word. If at least one digital audio signal contains a wake-up word, it indicates that the electronic device needs to be woken up. To prevent this from being caused by a light wave attack, step S203 is executed. If none of the u digital audio signals contain a wake-up word, the processor continues to determine whether the received u digital audio signals contain a wake-up word.

[0147] S203, Obtain the reference energy value D of n pickups. max0 .

[0148] If it is determined that at least one digital audio signal in the digital audio signal converted from the analog-to-digital converter (A / D converter) of each of the u microphones contains a wake-up word, the reference energy value D of the n microphones can be determined. max0For example, the reference energy value D can be determined based on the energy value of the digital audio signal converted from an A / D converter for each of the n microphones. max0 For example, the highest energy value among the energy values ​​of the digital audio signal after AD conversion for each of the n microphones can be selected as the reference energy value D. max0 Alternatively, the mean or variance of any two or more energy values ​​from the digital audio signal obtained after AD conversion for each of the n microphones can be used as the reference energy value D. max0 No restrictions are imposed here.

[0149] One possible implementation is to calculate the energy value of each of the n microphones based on the digital audio signal converted from an analog-to-digital converter (A / D) for each of the n microphones. Specifically, a second energy value can be calculated for each of the u microphones based on the digital audio signal converted from an A / D converter for each of the u microphones, thus obtaining u second energy values. For example, the digital audio signal converted from an A / D converter for each of the u microphones includes the aforementioned D... p (q) and information indicating that the sampling point is the qth sampling point. Based on the above digital audio signal, the processor can calculate the second energy value corresponding to each microphone using the following formula (1), and then obtain u second energy values.

[0150]

[0151] Among them, D p Let be the second energy value corresponding to the p-th pickup out of u pickups, M be the total number of sampling points, and D be the second energy value. p (q) is the digital signal value of the amplitude of the q-th sampling point corresponding to the p-th microphone among the u microphones after AD conversion. In the first method described above, M in formula (1) is greater than or equal to m. In the second method described above, M = m in formula (1). In the second method described above, S201 is essentially a calculation of the digital audio signal corresponding to each of the u microphones in the same sampling period (e.g., the first sampling period).

[0152] Alternatively, the third energy value corresponding to each of the (nu) pickups can be calculated based on the digital audio signal converted from the analog-to-digital converter to the corresponding digital audio signal of each of the (nu) pickups, thus obtaining (nu) third energy values. Here, (nu) pickups are the pickups other than u pickups among the n pickups. For example, after obtaining the digital audio signal converted from the analog-to-digital converter to the corresponding digital audio signal of each of the (nu) pickups, the processor can calculate the third energy value corresponding to each pickup using the following formula (2), thereby obtaining (nu) third energy values.

[0153]

[0154] Among them, D r Let be the third energy value corresponding to the r-th pickup out of (nu) pickups, M be the total number of sampling points, and D be the third energy value. r (q) represents the amplitude value V at the q-th sampling point corresponding to the r-th pickup among (nu) pickups. r (q) The digital signal value after A / D conversion. Where r is a positive integer greater than or equal to 1 and less than or equal to (nu).

[0155] Furthermore, after obtaining u second energy values ​​and (nu) third energy values, the reference energy values ​​D of n pickups can be obtained based on the u second energy values ​​and (nu) third energy values. max0 .

[0156] For example, after obtaining u second energy values, the largest second energy value D can be obtained according to the following formula (3). max1 Alternatively, the largest second energy value D can be obtained by sorting, for example, by sorting the u second energy values ​​from largest to smallest or smallest to largest. max1 .

[0157] D max1 =max 1≤p≤u D p Equation (3)

[0158] After obtaining (nu) third energy values, the largest third energy value D can be obtained according to the following formula (4). max2 Alternatively, the largest third energy value D can be obtained by sorting, for example, by sorting the (nu) third energy values ​​from largest to smallest or smallest to largest. max2 .

[0159] D max2 =max 1≤r≤(n-u) D r Equation (4)

[0160] The second energy value D is obtained. max1 and the third energy value D max2 Then, the second energy value D was calculated. max1 and the third energy value D max2 The maximum value in the range is the reference energy value D. max0 .

[0161] Optionally, the reference energy value D is obtained. max0 Then, considering that some kind of error might cause the obtained baseline energy value D to be incorrect... max0 The calculation has a problem; to ensure accuracy, the baseline energy value D can be adjusted first. max0 Make a judgment. For example, the reference energy value D can be used. max0 The upper limit of energy value D corresponding to the pre-set pickup th0 For comparison, when the reference energy value D max0 Greater than the upper limit of energy value D th0 This indicates the reference energy value D max0 If there is an error, the baseline energy value D can be recalculated based on the sampling results. max0 Alternatively, an energy value less than the upper limit D can be selected. th0 The largest energy value among all energy values ​​is used as the baseline energy value D. max0 Furthermore, the reference energy value D can also be adjusted in subsequent steps. max0 Make a judgment.

[0162] S204, Based on the reference energy value D max0 For each of the u second energy values ​​corresponding to u pickups, evaluate D. max0 The deviation between each of the u second energy values ​​is used to obtain u deviations.

[0163] For example, according to D max0 For each of the u second energy values, we obtain u deviations. Specifically, the p-th deviation b corresponding to the p-th pickup... p It can be calculated using the following formula (5).

[0164]

[0165] Among them, D max0 D p Please refer to the previous description; it will not be repeated here.

[0166] Alternatively, the p-th deviation b corresponding to the p-th pickup p It can be calculated using the following formula (6).

[0167] b p =Dmax0 -D p Equation (6)

[0168] It should be noted that the above formulas (5) and (6) are only illustrative examples, and the p-th deviation b is not explicitly stated. p The calculation formulas include, but are not limited to, the above formulas (5) and (6). Any formula capable of evaluating D... max0 With D p The formulas relating the deviations between them are all within the scope of this application.

[0169] S205. Based on u deviations, determine whether it is a suspected light wave attack.

[0170] After obtaining u deviations, each deviation is compared with a preset range to determine whether it is a suspected light wave attack. It is understood that when none of the microphones are subjected to a light wave attack, the analog audio signals collected by each microphone are relatively similar, and therefore the energy values ​​corresponding to each microphone are also relatively similar. Thus, all u deviations are within the preset range. In other words, if each of the u deviations is within the preset range, it can be determined that no suspected light wave attack has occurred, and the electronic device can be woken up, i.e., S206 is executed. If at least one of the u deviations is outside the preset range, it can be determined that a suspected light wave attack has occurred, and S207 can be executed. For example, the preset range includes one of the following: [0, b0], [0, b0), (0, b0), (0, b0]; where b0 is a preset second threshold. For example, the second threshold b0 can be 0.9. Of course, the second threshold b0 can also be other values, which are not limited here.

[0171] S206, Wake up the electronic device.

[0172] S207. Determine whether a microphone with abnormal pickup capability has been pre-marked.

[0173] Since microphones with abnormal pickup capabilities have been pre-identified and identified in S1, it is now possible to determine from the pre-stored data whether there are any microphones with abnormal pickup capabilities. For example, the processor can read data stored in memory to determine if there are any microphones with abnormal pickup capabilities.

[0174] When all n microphones have normal pickup capabilities, it means that they can all collect audio signals normally. However, if the energy values ​​of the audio signals collected by two microphones differ significantly, it can be determined that one of the microphones failed to collect the audio signal normally. At this point, it can be determined that the electronic device has been attacked by light waves, so it is necessary to intercept this wake-up. At this time, an alarm message can be output and the electronic device can not be woken up, i.e., S208 is executed.

[0175] When one of the n microphones has a malfunctioning pickup capability, that microphone cannot properly acquire the audio signal. This can lead to a situation where the energy values ​​of the audio signals acquired by two microphones differ significantly. Therefore, to determine whether the malfunction is caused by a microphone with a malfunctioning pickup capability, step S209 can be executed.

[0176] S208: Output alarm information without waking up electronic devices.

[0177] Once an electronic device is identified as being under a light wave attack, it can output an alarm message to notify the user. For example, the alarm message can be text, graphics, or a combination of both. For example, the electronic device can send the alarm message to the user's mobile terminal via a wired or wireless network. The user can pre-bind their mobile terminal to the electronic device so that the device can know the target address when outputting the alarm message. For example, the electronic device can also output the alarm message itself, such as through sound, vibration, or a combination of both. Furthermore, when the electronic device has a display screen, it can also display text, graphics, or a combination of both on the screen as alarm messages. In addition, while outputting the alarm message, the electronic device can also remain unactivated (e.g., not respond to the wake-up call) to enhance security.

[0178] S209. Based on the microphone with abnormal pickup capability, determine whether it has been attacked by light waves.

[0179] When a microphone with abnormal pickup capability is pre-marked, the presence of this microphone can be used to determine whether a light wave attack has actually occurred. If a light wave attack is confirmed, the wake-up call is intercepted, i.e., step S208 is executed. If no light wave attack is confirmed, the electronic device is woken up, i.e., step S206 is executed.

[0180] For example, the specific steps of S205 can be as follows: Figure 6 As shown in (B), specifically, S205 may include:

[0181] S2051. Determine whether all u deviations are within the preset range.

[0182] By comparing each of the u deviations with a preset range, it can be determined whether all u deviations are within the preset range. Specifically, if all u deviations are within the preset range, it can be determined that there is no suspected light wave attack, and step S2052 is executed; if at least one of the u deviations is outside the preset range, it can be determined that there is a suspected light wave attack, and step S2053 is executed.

[0183] S2052, suspected of not being attacked by light waves.

[0184] S2053, suspected of being attacked by light waves.

[0185] For example, the specific steps of S209 can be as follows: Figure 6 As shown in (C1), specifically, S209 may include:

[0186] S20911. Select s pickups from u pickups that are different from the pre-marked pickups with abnormal pickup capabilities, thereby obtaining s deviations corresponding to the s pickups, where s < u.

[0187] By comparing the identifiers of the u pickups with the pre-marked pickups with abnormal pickup capabilities, s pickups that are different from the pre-marked pickups with abnormal pickup capabilities can be selected from the u pickups, where s < u. Since each pickup in the u pickups corresponds to one of the u deviations, obtaining the s pickups from the u pickups yields the s deviations.

[0188] S20912. Determine whether any of the s deviations are outside the preset range.

[0189] By comparing each of the s deviations with a preset range, it can be determined whether any of the m deviations fall outside the preset range, where m is greater than or equal to 0 and less than or equal to s. Since the energy values ​​of microphones with abnormal pickup capabilities have been eliminated, the pickup capabilities of each microphone corresponding to the s deviations are normal. Therefore, if any of the s deviations fall outside the preset range, it can be determined that a light wave attack has indeed occurred, and S20913 is executed; otherwise, S20914 is executed.

[0190] S20913, Attacked by light waves.

[0191] S20914, Not attacked by light waves.

[0192] The specific steps of the exemplary S209 can also be as follows Figure 6 As shown in (C2), specifically, S209 may include:

[0193] S20921. Based on the u deviations corresponding to the u pickups, obtain v pickups with abnormal pickup capabilities from the u pickups, where 1≤v<u.

[0194] Each of the u deviations corresponding to the u pickups is compared with the first threshold a0 described in S1 above, thus identifying v pickups among the u pickups that have abnormal pickup capabilities; where v < n, and v is a positive integer greater than or equal to 1. It is understandable that when the pickup capabilities of all pickups are normal, the analog audio signals collected by each pickup are relatively similar, and therefore the energy values ​​corresponding to each pickup are also relatively similar. Thus, the deviation corresponding to each pickup is less than or equal to the first threshold a0. That is, when the deviation corresponding to the w-th pickup is greater than the first threshold a0, it can be determined that the w-th pickup has an abnormal pickup capability. At this point, the pickup capability of the w-th pickup can be marked as abnormal, or the w-th pickup can be marked as an abnormal pickup.

[0195] S20922. Determine whether the pre-marked microphones with abnormal pickup capabilities include v microphones with abnormal pickup capabilities.

[0196] By comparing the identifiers of the v microphones with abnormal pickup capabilities with the pre-marked identifiers of microphones with abnormal pickup capabilities, it can be determined whether the pre-marked list of microphones with abnormal pickup capabilities includes the v microphones with abnormal pickup capabilities. If the pre-marked list of microphones with abnormal pickup capabilities includes the v microphones with abnormal pickup capabilities, it indicates that the large difference in the energy values ​​of the audio signals collected by the two microphones is caused by the microphones with abnormal pickup capabilities. In this case, it can be determined that there is no light wave attack, and S20923 is executed. If the pre-marked list of microphones with abnormal pickup capabilities does not include the v microphones with abnormal pickup capabilities, it indicates that the large difference in the energy values ​​of the audio signals collected by the two microphones is not caused by the microphones with abnormal pickup capabilities. In this case, it can be determined that there is a light wave attack, and S20924 is executed.

[0197] S20923, Not attacked by light waves.

[0198] S20924, Not attacked by light waves.

[0199] For example, the specific steps of S20921 can be as follows: Figure 6 As shown in (D). Specifically, S20921 may include:

[0200] S209211. Determine whether the w-th deviation corresponding to the w-th pickup among u pickups is greater than the first threshold a0.

[0201] By comparing the w-th deviation of the w-th pickup among u pickups with the first threshold a0, the magnitude of the difference between the two can be determined. When the w-th deviation is less than or equal to the first threshold a0, it indicates that the energy value D corresponding to the w-th pickup is... wCompared with the reference energy value D max0 The small deviation between them indicates that the pickup capability of the w-th pickup is normal, i.e., S209212 is executed.

[0202] When the w-th deviation is greater than the first threshold a0, it indicates that the energy value D corresponding to the w-th pickup is... w Compared with the reference energy value D max0 The deviation between them is small, indicating that the pickup capability of the w-th pickup is abnormal. At this time, S209213 can be executed.

[0203] Optionally, considering the possibility that the obtained reference energy value D might be affected by some error, max0 There may be errors or other issues; to ensure accuracy, we can first determine D. max0 Is it greater than the preset energy value limit D? th0 .

[0204] D max0 With the preset energy value limit D th0 Compare and determine the size of the two. Where, when D... max0 Less than or equal to the upper limit of energy value D th0 When, it indicates D max0 Normally, at this point it can be determined that the pickup capability of the w-th pickup is abnormal, i.e., execute S210213. When D max0 Greater than the upper limit of energy value D th0 When, it indicates D max0 An error has occurred; at this point, S203 and all subsequent processes can be re-executed.

[0205] S209212, The pickup capability of the w-th pickup is normal; w = w + 1, and determine whether the updated w is less than or equal to u.

[0206] At this point, there's no need to mark the w-th pickup as an abnormal pickup, or to mark its pickup capability as abnormal. Then, update w by setting w = w + 1. Check if the updated w is less than or equal to u. If it is, execute step S209211. If it is not less than or equal to u, end the process, record which pickups have abnormal pickup capabilities, or record which pickups are abnormal, thus obtaining v pickups with abnormal pickup capabilities.

[0207] S209213, The pickup capability of the w-th pickup is abnormal; w = w + 1, and determine whether the updated w is less than or equal to u.

[0208] At this point, the w-th pickup is marked as an abnormal pickup, or its pickup capability is marked as abnormal. Then, w is updated by setting w = w + 1. It is then determined whether the updated w is less than or equal to u. If it is less than u, step S209211 is executed. If it is greater than u, the process ends, and the records of which pickups have abnormal pickup capabilities, or which pickups are abnormal, are recorded, thus obtaining v pickups with abnormal pickup capabilities.

[0209] Based on the methods described in the above embodiments, this application also provides a chip. Please refer to... Figure 7 , Figure 7 This is a schematic diagram of a chip structure provided in an embodiment of this application. Figure 7 As shown, chip 700 includes one or more processors 701 and interface circuitry 702. Optionally, chip 700 may also include a bus 703. Wherein:

[0210] The processor 701 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the processor 701 or by software instructions. The processor 701 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods and steps disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The interface circuit 702 can be used to send or receive data, instructions, or information. The processor 701 can process the data, instructions, or other information received by the interface circuit 702 and send the processed information out through the interface circuit 702.

[0211] Optionally, the chip also includes memory, which may include read-only memory and random access memory, providing operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM). Optionally, the memory stores executable software modules or data structures, and the processor can perform corresponding operations by calling operation instructions stored in the memory (which may be stored in the operating system).

[0212] Optionally, the interface circuit 702 can be used to output the execution results of the processor 701.

[0213] It should be noted that the functions of the processor 701 and the interface circuit 702 can be implemented through hardware design, software design, or a combination of hardware and software; no restrictions are imposed here.

[0214] It should be understood that the steps of the above method embodiments can be implemented by hardware logic circuits or software instructions in a processor. The chip can be applied in the above-described electronic device 200 to implement the methods provided in the embodiments of this application.

[0215] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0216] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. One exemplary embodiment couples a storage medium to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0217] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0218] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

Claims

1. A method for preventing light wave attacks, characterized in that, The method is applied to an electronic device; the electronic device includes n microphones; the method includes: An audio signal containing a wake-up word is acquired through u of the n microphones; If the energy value corresponding to at least one of the u microphones deviates from the reference energy value of the n microphones outside a preset range, and the reference energy value is less than or equal to a preset upper limit value, the electronic device outputs an alarm message, and the electronic device is not woken up by the audio signal. The electronic device is awakened by the audio signal when the deviation between the energy value corresponding to each of the u microphones and the reference energy value of the n microphones is within a preset range. Where n is a positive integer greater than or equal to 1; u is a positive integer greater than or equal to 1 and less than n.

2. The method according to claim 1, characterized in that, Before acquiring an audio signal containing a wake-up word through u of the n microphones, the method further includes: obtaining that k of the n microphones are abnormal; Before the electronic device outputs alarm information and the electronic device is not woken up by the audio signal, the method further includes: at least one of the u microphones is not located within the k microphones, nor does it overlap with the k microphones; Where k is a positive integer greater than or equal to 0 and less than n.

3. The method according to claim 2, characterized in that, The electronic device further includes an audio module and a processor; the n microphones are connected to the processor through the audio module; the audio module is used to sample and convert the analog audio signal transmitted from any one of the n microphones to digital, and transmit the converted digital signal to the processor. The energy value corresponding to each of the u microphones is calculated as follows: The number of sampling points corresponding to each of the u microphones is M, and the amplitude value of the q-th sampling point among the M sampling points corresponding to the p-th microphone is... , The digital signal value after analog-to-digital conversion is According to formula (1), the energy value corresponding to the p-th pickup among u pickups is calculated. ; Formula (1) Where p is a positive integer greater than or equal to 1 and less than or equal to u; q is a positive integer greater than or equal to 1 and less than or equal to M.

4. The method according to claim 3, characterized in that, The reference energy value is calculated in the following manner: The number of sampling points corresponding to each of the (nu) pickups is M. The (nu) pickups are the remaining pickups from the n pickups excluding the u pickups. The amplitude value of the q-th sampling point among the M sampling points corresponding to the r-th pickup in the (nu) pickups is... , The digital signal value after analog-to-digital conversion is According to formula (2), the energy value corresponding to the r-th pickup among the (nu) pickups is calculated. ; Formula (2) Where r is a positive integer greater than or equal to 1 and less than or equal to (nu); According to formula (3), the maximum energy value of u energy values ​​is calculated. ; Formula (3) According to formula (4), the maximum energy value of (nu) energy values ​​is calculated. ; Equation (4) according to and Calculations yielded and The maximum value in the range is the reference energy value. .

5. The method according to claim 4, characterized in that, The deviation between the energy value of at least one of the u pickups and the reference energy value of the n pickups is calculated as follows: Equation (5) in, The deviation is described above.

6. The method according to claim 4, characterized in that, The deviation between the energy value of at least one of the u pickups and the reference energy value of the n pickups is calculated as follows: Equation (6) in, The deviation is described above.

7. The method according to claim 5 or 6, characterized in that, The preset range includes one of the following: [0, ],[0, (0, (0, ];in, This is the preset second threshold.

8. The method according to any one of claims 3-6, characterized in that, The fact that k out of the n microphones are malfunctioning was obtained through the following method: The audio module receives another analog audio signal collected by the n microphones, and after sampling and analog-to-digital conversion, obtains another digital signal after analog-to-digital conversion; Based on the analog-to-digital converted digital signal corresponding to each of the n microphones, the first energy value corresponding to each microphone is calculated, thereby obtaining n first energy values; Based on the n first energy values, the maximum value of the n first energy values ​​is obtained. ; According to the above And for each of the n first energy values, calculate the... Another deviation between each of the first energy values, thus obtaining n other deviations; Based on the n other deviations, it is determined that k of the n pickups are abnormal.

9. The method according to claim 8, characterized in that, The n first energy values ​​are obtained in the following way: According to formula (1), the first energy value corresponding to the i-th microphone among the n microphones is calculated. ; Official (1) in, Let M be the first energy value corresponding to the i-th microphone among the n microphones, and M be the total number of sampling points for the i-th microphone. The amplitude value of the sampling point corresponding to the i-th microphone among the n microphones is the digital signal value after analog-to-digital conversion.

10. The method according to claim 9, characterized in that, The maximum value of the n first energy values ​​is obtained based on the n first energy values. It was obtained in the following way: According to formula (2), the maximum value of the n first energy values ​​is obtained. ; Official (2).

11. The method according to claim 10, characterized in that, According to the above And for each of the n first energy values, calculate the... Another deviation from each of the first energy values ​​is obtained as follows: Official (3) in, This refers to the i-th deviation corresponding to the i-th pickup among the n pickups.

12. The method according to claim 10, characterized in that, According to the above And for each of the n first energy values, calculate the... Another deviation from each of the first energy values ​​is obtained as follows: Official (4) in, This refers to the i-th deviation corresponding to the i-th pickup among the n pickups.

13. The method according to claim 11 or 12, characterized in that, Based on the n additional deviations, it was determined that k of the n pickups were faulty, which was obtained in the following way: Of the n additional deviations, the one greater than a preset first threshold is... The number of the other deviation is k.

14. The method according to any one of claims 1-6 or 9-12, characterized in that, The n pickups include, in whole or in part, at least one of the following: microphones, microphone arrays.

15. An electronic device comprising n microphones, an audio module, a memory, and a processor; the n microphones being connected to the processor via the audio module; the audio module being configured to sample, convert analog audio signals transmitted from any one of the n microphones to digital signals, and transmit the converted digital signals to the processor; the memory storing a computer program; and when the computer program is executed by the processor, causing the electronic device to perform the method as described in any one of claims 1-14.

16. A computer-readable storage medium storing a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1-14.

17. A computer program product, characterized in that, When the computer program product is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1-14.

Images