Microphone switching method and electronic device

Abstract

The application discloses a microphone switching method and electronic equipment, and relates to the field of audio, and is used for switching a microphone in a microphone immersion liquid scene. The microphone switching method comprises the following steps: in each detection period, performing feature extraction on a first sound signal and a second sound signal to obtain frequency domain information of the first sound signal and frequency domain information of the second sound signal; calculating a model coincidence degree of the first sound signal according to the frequency domain information of the first sound signal and frequency domain information of an immersion liquid microphone model; if the model coincidence degree of the first sound signal is greater than or equal to a model coincidence degree threshold value, and the frequency domain information of the first sound signal meets a condition, then increasing a cumulative number; after a plurality of detection periods, if the cumulative number is greater than a cumulative threshold value, then switching the second microphone to a main microphone and switching the first microphone to an auxiliary microphone.

Description

Technical Field

[0001] This application relates to the field of audio, and more particularly to a microphone switching method and an electronic device. Background Technology

[0002] If the microphone of an electronic device such as a mobile phone is immersed in liquid, it can lead to problems such as low recording volume, noise, and microphone short circuit failure. Therefore, many electronic devices have waterproof designs, such as using a waterproof and breathable membrane to protect the microphone. The main disadvantage of this method is that the cost of the waterproof and breathable membrane is higher than the cost of the microphone device itself, and the waterproof and breathable membrane is easily damaged by stress or airflow, which can easily cause it to fail in its protective function for the microphone.

[0003] This solution proposes an optimized design for microphone immersion scenarios. When a random microphone fails due to user scenarios, a software solution can be used to switch to a working microphone, ensuring that there are no problems for actual user use. Summary of the Invention

[0004] This application provides a microphone switching method and an electronic device for switching microphones in a scenario where the microphone is immersed in liquid.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, a microphone switching method is provided, applied to an electronic device including a first microphone and a second microphone, wherein the first microphone is a main microphone. The method includes: in each detection cycle, performing feature extraction on a first sound signal to obtain frequency domain information of the first sound signal, and performing feature extraction on a second sound signal to obtain frequency domain information of the second sound signal; the first sound signal is a sound signal acquired through the first microphone, and the second sound signal is a sound signal acquired through the second microphone; in each detection cycle, calculating the model compliance degree of the first sound signal based on the frequency domain information of the first sound signal and the frequency domain information of the immersion microphone model; in each detection cycle, determining whether the frequency domain information of the first sound signal meets a condition; in each detection cycle, if the model compliance degree of the first sound signal is greater than or equal to a model compliance degree threshold, and the frequency domain information of the first sound signal meets the condition, then increasing the cumulative count; after multiple detection cycles, if the cumulative count is greater than the cumulative threshold, then switching the second microphone to the main microphone and switching the first microphone to the auxiliary microphone.

[0007] The microphone switching method provided in this application collects a first sound signal through a first microphone and a second microphone, with the first microphone being the primary microphone. In each detection cycle, if the model compliance of the first sound signal relative to the liquid-immersed microphone model is greater than or equal to a model compliance threshold, and the frequency domain information of the first sound signal meets the conditions, the cumulative count is increased until it exceeds the cumulative threshold. Then, the second microphone is switched to the primary microphone, and the first microphone is switched to the secondary microphone. This achieves microphone switching even when the microphone is immersed in liquid.

[0008] In one possible implementation, the method further includes reducing the accumulated count if the model compliance of the first audio signal is less than a model compliance threshold, or if the frequency domain information of the first audio signal does not meet the conditions. If a user's finger blocks the first microphone, the finger will not remain blocked. After a period of time, the user's finger will leave the first microphone, and the microphone will no longer be blocked. However, if the first microphone is immersed in liquid, the accumulated count can be reduced if the first microphone is not blocked in the current detection cycle. For example, the accumulated count can be reduced by 1 in each detection cycle, down to a minimum of 0. The purpose is to avoid erroneous microphone switching due to user blocking of the microphone.

[0009] In one possible implementation, the model conformance of the first sound signal is calculated based on the frequency domain information of the first sound signal and the frequency domain information of the immersion microphone model. This includes comparing the frequency domain information of the first sound signal with the frequency domain information of the immersion microphone model at multiple first target frequencies to calculate the model conformance of the first sound signal. The model conformance is the degree of similarity between the frequency domain information of the first sound signal and the frequency domain information of the immersion microphone model.

[0010] In one possible implementation, there are M first target frequencies, and at the i-th first target frequency f i Below, the frequency domain information of the first sound signal is A1(f i The frequency domain information of the immersion microphone model is H(f). i The model conformity of the first sound signal is ). 1≤i≤M. Select M representative first target frequencies of the liquid-immersed microphone model, compare the frequency domain information of the first sound signal with the frequency domain information of the liquid-immersed microphone model, and thus determine the model conformity of the first sound signal.

[0011] In one possible implementation, at the j-th second target frequency f j Below, the frequency domain information of the first sound signal is A1(f j The frequency domain information of the second sound signal is A2(f j The frequency domain information of the first sound signal satisfies the condition. and, 1≤j≤N,Th err For the differential threshold, Th abs This is the absolute threshold.

[0012] The first microphone, due to liquid immersion, experienced a microphone-blocking effect, resulting in a lower frequency response in the low-frequency range of the first audio signal. The frequency response of the first audio signal differed significantly from that of the second audio signal in the low-frequency range. The high-frequency noise caused by the liquid immersion superimposed on the frequency response of the first audio signal, making the high-frequency response of the first audio signal less smooth. However, the frequency response of the first and second audio signals remained relatively similar in the high-frequency range. The low-frequency range is what causes... The main reason.

[0013] Furthermore, if the first microphone is functioning correctly and has captured ambient noise, the frequency domain information of the ambient noise will be superimposed on both the high-frequency and low-frequency components of the first audio signal, causing the overall frequency domain information of the first audio signal to be too high, which does not meet the requirements. so This eliminates the possibility that the first microphone is functioning correctly and is picking up ambient noise.

[0014] In one possible implementation, in each detection cycle, the model conformity of the first sound signal is calculated based on the frequency domain information of the first sound signal and the frequency domain information of the liquid-immersed microphone model. This includes: if at least a portion of the frequency domain information of the first sound signal is greater than a preset threshold, then in each detection cycle, the model conformity of the first sound signal is calculated based on the frequency domain information of the first sound signal and the frequency domain information of the liquid-immersed microphone model. After excluding the ruptured membrane microphone scenario, it can be determined whether it is a liquid-immersed microphone scenario.

[0015] In one possible implementation, the method further includes: if the frequency domain information of the first sound signal is all less than a preset threshold, then the second microphone is switched to the main microphone and the first microphone is switched to the auxiliary microphone. When the diaphragm of the first microphone is ruptured due to external force, airflow, etc., the frequency domain information of the first sound signal collected is very small because the amplitude of diaphragm vibration is reduced. Therefore, the first microphone is switched to the auxiliary microphone at this time, and the sound signal collected by the second microphone is clearer.

[0016] In a second aspect, an electronic device is provided, including a processor and a memory, wherein instructions are stored in the memory, and when the processor executes the instructions, the electronic device performs the method as described in the first aspect and any embodiment thereof.

[0017] Thirdly, a computer-readable storage medium is provided that stores instructions which, when executed on an electronic device, cause the electronic device to perform the method as described in the first aspect and any embodiment thereof.

[0018] Fourthly, a computer program product is provided, including instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in the first aspect and any embodiment thereof.

[0019] The technical effects of the second and fourth aspects refer to the technical effects of the first aspect and any of its embodiments, and will not be repeated here. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of the appearance of an electronic device provided in an embodiment of this application;

[0022] Figure 3 A schematic diagram of a settings interface provided in an embodiment of this application;

[0023] Figure 4 A schematic flowchart illustrating a microphone switching method provided in an embodiment of this application;

[0024] Figure 5 A flowchart illustrating another microphone switching method provided in an embodiment of this application;

[0025] Figure 6 A schematic diagram of the frequency domain information of a diaphragm microphone model provided in an embodiment of this application;

[0026] Figure 7 A time-domain schematic diagram of high-frequency noise provided for an embodiment of this application;

[0027] Figure 8 A schematic diagram of the frequency domain information of a liquid-immersed microphone model provided in an embodiment of this application;

[0028] Figure 9 A schematic diagram illustrating the frequency domain information of a first sound signal and a second sound signal provided in an embodiment of this application;

[0029] Figure 10 This is a time-domain schematic diagram illustrating a sound signal from a microphone being immersed in liquid, provided as an embodiment of this application, where a user blocks the microphone with their finger. Detailed Implementation

[0030] First, some concepts involved in this application will be described.

[0031] The terms "first" and "second" used in the embodiments of this application are only used to distinguish features of the same type and should not be construed as indicating relative importance, quantity, order, etc.

[0032] The terms "exemplary" or "for example" used in the embodiments of this application are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0033] The terms "coupling" and "connection" used in the embodiments of this application should be interpreted broadly. For example, they can refer to a physical direct connection or an indirect connection achieved through electronic devices, such as a connection achieved through resistors, inductors, capacitors or other electronic devices.

[0034] like Figure 1 As shown, this application embodiment provides an electronic device 101, which is an electronic device with wireless communication capabilities. The electronic device can be mobile or fixed. It can be deployed on land (e.g., indoors or outdoors, handheld or vehicle-mounted), on water (e.g., on ships), or in the air (e.g., airplanes, balloons, and satellites). This electronic device can be referred to as user equipment (UE), access terminal, terminal unit, subscriber unit, terminal station, mobile station (MS), mobile station, terminal agent, or terminal device, etc. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, smart bracelet, smart screen, smartwatch, virtual reality (VR) device, augmented reality (AR) device, terminal in industrial control, terminal in self-driving, terminal in remote medical care, terminal in smart grid, terminal in transportation safety, terminal in smart city, terminal in smart home, etc. This application does not limit the specific type and structure of the electronic device. One possible structure of the electronic device is described below.

[0035] Taking mobile phones as an example, Figure 1A possible structure of an electronic device 101 is shown. This electronic device 101 may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a power management module 240, a battery 241, a wireless charging coil 242, antenna 1, antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, a sensor module 280, buttons 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc. Optionally, in some embodiments, it may also include an audio digital signal processor (ADSP) 243.

[0036] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 101. In other embodiments of this application, the electronic device 101 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0037] Processor 210 may include one or more processing units, such as: a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a system-on-chip (SoC), a central processing unit (CPU), an application processor (AP), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, and a neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors. For example, processor 210 may be an application processor (AP). Alternatively, processor 210 may be integrated into a system-on-chip (SoC). Or, processor 210 may be integrated into an integrated circuit (IC) chip. The processor 210 may include an analog front end (AFE) and a micro-controller unit (MCU) in an IC chip.

[0038] The processor 210 executes the antenna feeding control method provided in this application embodiment by executing the program and computer instructions stored in the internal memory 221.

[0039] The processor 210 may also include a memory for storing computer instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can store computer instructions or data that the processor 210 has just used or that are being used repeatedly. If the processor 210 needs to use the same computer instructions or data again, it can retrieve them directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves system efficiency.

[0040] In some embodiments, the processor 210 may include one or more interfaces. These interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, etc.

[0041] The ADSP 243 can be coupled to the audio module 270 and the sensor module 280. The ADSP 243 can process audio signals and sensor data. Even when the processor is in sleep mode, the ADSP 243 can continue operating, thereby reducing the power consumption of the electronic device.

[0042] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 101. In other embodiments of this application, the electronic device 101 may also adopt different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0043] The external storage interface 220 can be used to connect an external memory card, such as a micro SanDisk (Micro SD) card, to expand the storage capacity of the electronic device 101. The external memory card communicates with the processor 210 through the external storage interface 220 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0044] Internal memory 221 can be used to store computer executable program code, which includes computer instructions. Processor 210 executes various functional applications and data processing of electronic device 101 by running the computer instructions stored in internal memory 221. In addition, internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0045] The memory involved in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0046] Electronic device 101 can implement audio functions such as music playback and recording through audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone jack 270D, and application processor.

[0047] Audio module 270 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. In some embodiments, audio module 270 may be located in processor 210, or some functional modules of audio module 270 may be located in processor 210. Speaker 270A, also called a "loudspeaker," is used to convert audio electrical signals into sound signals. Receiver 270B, also called a "handpiece," is used to convert audio electrical signals into sound signals. Microphone 270C, also called a "microphone" or "microphone," is used to convert sound signals into electrical signals. Electronic device 101 may be equipped with at least one microphone 270C. Headphone jack 270D is used to connect wired headphones. Headphone jack 270D may be a USB interface 230, or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

[0048] Buttons 290 include a power button, volume buttons, etc. Buttons 290 can be mechanical buttons or touch buttons. Electronic device 101 can receive button input and generate key signal inputs related to user settings and function control of electronic device 101. Motor 291 can generate vibration alerts. Motor 291 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 292 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 295 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 295 to achieve contact and separation with electronic device 101. Electronic device 101 can support one or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 295 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. In some embodiments, electronic device 101 employs an embedded SIM (eSIM) card, which can be embedded in electronic device 101 and cannot be separated from electronic device 101.

[0049] Electronic device 101 can implement shooting functions through an ISP, camera 293, video codec, GPU, display 294, and application processor. The ISP is used to process data fed back from the camera 293. In some embodiments, the ISP can be located within the camera 293. The camera 293 is used to capture still images or videos. In some embodiments, electronic device 101 may include one or N cameras 293, where N is a positive integer greater than 1, for example... Figure 2 The front-facing camera 2931 and the rear-facing camera 2932 are shown.

[0050] Electronic device 101 can implement display functions through a GPU, display screen 294, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 210 may include one or more GPUs, which execute computer instructions to generate or modify display information.

[0051] The sensor module 280 may include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer, a distance sensor, a proximity sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, an angle sensor, etc. When the display screen 294 is a foldable screen, the angle sensor can detect the folding angle of the display screen 294, which ranges from 0 to 180 degrees.

[0052] Battery 241 may include one or more batteries to power a load. Power management module 240 receives charging input from a charger. The charger may be a wireless charger, such as a wireless charging dock, another electronic device 101 with reverse wireless charging capability, etc. Power management module 240 may receive wireless charging input via the wireless charging coil 242 of the electronic device. The charger may also be a wired charger; for example, power management module 240 may receive charging input from a wired charger via USB interface 230. Power management module 240 is also referred to as a charging chip.

[0053] The power management module 240 charges the battery 241 while simultaneously supplying power to the electronic devices. It receives input from the battery 241 and powers the processor 210, internal memory 221, external memory interface 220, display screen 294, camera 293, and wireless communication module 260. The power management module 240 can also monitor parameters such as the battery 241's capacity, voltage, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 240 may also be integrated into the processor 210.

[0054] The display screen 294 is used to display images, videos, etc. The display screen 294 includes a display panel. In some embodiments, the electronic device 101 may include one or more display screens 294.

[0055] The wireless communication function of electronic device 101 can be realized through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor, etc.

[0056] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 101 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0057] The mobile communication module 250 (also known as the cellular communication module) can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on the electronic device 101. The wireless communication module 260 (including the satellite communication module) can provide wireless communication solutions, including wireless local area networks (WLANs) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS) (such as BeiDou satellite communication), satellite network communication (such as Tiantong satellite communication and Xingwang satellite communication), frequency modulation (FM), near-field communication (NFC), and infrared (IR) technologies, for use on the electronic device 101. In some embodiments, antenna 1 of the electronic device 101 is coupled to the mobile communication module 250, and antenna 2 is coupled to the wireless communication module 260, enabling the electronic device 101 to communicate with networks and other devices via wireless communication technology. The mobile communication module 250 and the wireless communication module 260 can be collectively referred to as radio frequency circuits.

[0058] like Figure 2 As shown in Figure A, the electronic device 101 may include a front-facing camera 2931 and a display screen 294. Figure 2 As shown in Figure B, the electronic device 101 may include a rear camera 2932. The front camera 2931 and the rear camera 2932 are used to capture still images or moving videos (collectively referred to as images). The display screen 294 is used to display images or receive user touch operations.

[0059] like Figure 2 As shown in Figures A and B, electronic device 101 includes microphone 2701 and microphone 2702, which are located at different positions on electronic device 101. For example, microphone 2701 is located at the top of electronic device 101, and microphone 2702 is located at the bottom of electronic device 101. It should be noted that this application uses electronic device 101 with two microphones as an example, but it is not intended to be limited to this, and it may include more microphones.

[0060] This application provides a microphone switching method. The trigger condition for the processor to execute the microphone switching method can be that the electronic device starts collecting sound signals (e.g., voice signals or human voice signals) through the microphone. For example, a user makes a call through the electronic device, the user opens a recording application to record sound, the user inputs text through a voice input method, or the user records sound through the voice sending function of a chat software. Figure 3 As shown, the electronic device can display a microphone switch 31 in the settings interface. When the microphone switch 31 is turned on by the user, the microphone switching method is allowed to be executed. When the microphone switch 31 is turned off by the user, the microphone switching method is not allowed to be executed.

[0061] like Figure 4 and Figure 5 As shown, the microphone switching method includes S101-S106, wherein, Figure 4 In this context, S102 and S103 are executed in parallel, for example, they can be executed through different threads, thereby improving the running speed. This is suitable for electronic devices with sufficient system resources, such as high-version electronic devices. Figure 5 In this process, S102 and S103 are executed sequentially. S102 can be executed first and then S103, or S103 can be executed first and then S102. For example, they can be executed through the same thread, which consumes less system resources and is suitable for electronic devices with limited system resources, such as low-version electronic devices.

[0062] S101. In each detection cycle, feature extraction is performed on the first sound signal to obtain the frequency domain information of the first sound signal, and feature extraction is performed on the second sound signal to obtain the frequency domain information of the second sound signal.

[0063] The first sound signal is a sound signal acquired through a first microphone, and the second sound signal is a sound signal acquired through a second microphone. For example, the first microphone can be... Figure 2 The microphone 2701 in the middle, the second microphone can be Figure 2 Microphone 2702 in the middle. Alternatively, the first microphone can be... Figure 2 The microphone 2702 in the middle, the second microphone can be Figure 2 Microphone 2701 in the middle.

[0064] In this embodiment, it is assumed that the first microphone is the primary microphone and the second microphone is the secondary microphone in the default state. The primary microphone is used to collect sound signals, while the secondary microphone is either inactive or used to collect background noise. For example, in a call scenario where sound is not played aloud, the microphone 2702, located at the bottom of the electronic device 101, is closer to the user's mouth and has a better voice collection effect, so microphone 2702 is the primary microphone and microphone 2701 is the secondary microphone. In a call scenario where sound is played aloud, since the speaker is generally also located at the bottom of the electronic device 101, microphone 2701 is the primary microphone and microphone 2702 is the secondary microphone to avoid excessive echo caused by the microphone 2702 collecting sound from the speaker.

[0065] The frequency domain information of a sound signal refers to its spectrum or frequency response. Feature extraction from sound can be performed as follows: A Fourier transform is applied to the first sound signal to obtain its spectrum, and a Fourier transform is applied to the second sound signal to obtain its spectrum. Alternatively, the energy of the second sound signal can be obtained by squaring its spectrum, and the energy of the first sound signal can be obtained by squaring its spectrum.

[0066] The detection cycle can be 10ms, 20ms, 30ms, 40ms, etc. The detection cycle is related to the improvement effect and can be adjusted according to the improvement effect.

[0067] If all frequency domain information of the first sound signal is less than a preset threshold (e.g., -111dB), then step S106 is executed according to the ruptured membrane microphone model, switching the second microphone to the main microphone and the first microphone to the auxiliary microphone, so that the sound signal collected by the second microphone is clearer. Otherwise, if at least some of the frequency domain information of the first sound signal is greater than the preset threshold, steps S102-S106 are executed according to the liquid immersion microphone model.

[0068] Examples such as Figure 6 The image shows the frequency domain information of a ruptured diaphragm microphone model. When the microphone diaphragm ruptures due to external forces, airflow, or other factors, the amplitude of the diaphragm vibration decreases, resulting in very low frequency domain information for the acquired sound signal, for example, all less than -110dB. Figure 7 As shown, unlike the ruptured membrane microphone model, when the microphone is immersed in liquid, it generates high-frequency noise due to short circuit. The frequency domain information of this high-frequency noise is close to that of normal speech, and it manifests as background noise. Figure 8 As shown, this is the frequency domain information of the diaphragm microphone model. High-frequency noise will exhibit high-frequency characteristics in the frequency domain. In the high-frequency part, the frequency domain information of high-frequency noise will be superimposed on the frequency domain information of the sound signal collected by the microphone, making the frequency domain information of the sound signal in the high-frequency part no longer smooth.

[0069] It should be noted that before proceeding to the next step, speech detection (or human voice detection) can be performed based on the frequency domain information of the first sound signal. If speech (or human voice) is detected, the next step can be performed; otherwise, the microphone switching method can be exited.

[0070] S102. In each detection cycle, the model conformity of the first sound signal is calculated based on the frequency domain information of the first sound signal and the frequency domain information of the immersion microphone model.

[0071] In one possible implementation, multiple first target frequencies can be selected. At each of these first target frequencies, the frequency domain information of the first sound signal is compared with the frequency domain information of the liquid-immersed microphone model to calculate the model fit of the first sound signal. The model fit is the degree of similarity between the frequency domain information of the first sound signal and the frequency domain information of the liquid-immersed microphone model.

[0072] Suppose there are M first target frequencies, and the i-th (1≤i≤M) first target frequency f i Below, the frequency domain information of the first sound signal is A1(f i The frequency domain information of the immersion microphone model is H(f). i The error ratio of the frequency domain information of the first sound signal is |A1(f) i )-H(f i )| / H(f i If the model conformity of the first sound signal is , then the model conformity of the first sound signal is .

[0073]

[0074] For example, in the liquid-immersion microphone model, the frequency domain information at 300Hz and 400Hz has the most prominent features (maximum frequency domain information), followed by the second harmonics at 300Hz and 400Hz (600Hz and 800Hz). Therefore, assuming there are four first target frequencies: 300Hz, 400Hz, 600Hz, and 800Hz, with f1 at 300Hz, f2 at 400Hz, f3 at 600Hz, and f4 at 800Hz, the model fit of the first sound signal is...

[0075] S103. In each detection cycle, determine whether the frequency domain information of the first sound signal meets the conditions.

[0076] Suppose there are N second target frequencies, and at the j-th (1≤j≤N) second target frequency f j Below, the frequency domain information of the first sound signal is A1(f j The frequency domain information of the second sound signal is A2(f jIf the sum of the frequency domain information of the first sound signal is..., then the sum of the frequency domain information of the first sound signal is... The sum of the frequency domain information of the second sound signal is The frequency domain information of the first sound signal meets the condition. and, The frequency domain information of the first sound signal does not meet the conditions. or, Th err For the differential threshold, Th abs This is an absolute threshold. The N second target frequencies can refer to frequencies across the entire frequency band, such as 1Hz-1000Hz.

[0077] The frequency domain information of the first sound signal meets the condition. and, The specific reasons are as follows:

[0078] For example, such as Figure 9 As shown, the frequency domain information of the second audio signal is relatively smooth. Due to liquid immersion, the first microphone experiences a similar "microphone jamming" effect, resulting in a lower frequency response in the low-frequency range of the first audio signal. The frequency domain information of the first and second audio signals differs significantly in the low-frequency range. The high-frequency noise caused by the liquid immersion superimposes on the frequency domain information of the first audio signal, making the frequency domain information of the first audio signal less smooth in the high-frequency range. However, the frequency domain information of the first and second audio signals does not differ much in the high-frequency range. The low-frequency range is what causes... The main reason.

[0079] Furthermore, if the first microphone is functioning correctly and has captured ambient noise, the frequency domain information of the ambient noise will be superimposed on both the high-frequency and low-frequency components of the first audio signal, causing the overall frequency domain information of the first audio signal to be too high, which does not meet the requirements. so This eliminates the possibility that the first microphone is functioning correctly and is picking up ambient noise.

[0080] Therefore, S103 is a judgment made based on the inherent characteristics of the frequency domain information of the first sound signal under the scenario of the first microphone being immersed in liquid.

[0081] S104. In each detection cycle, if the model compliance of the first sound signal is greater than or equal to the model compliance threshold, and the frequency domain information of the first sound signal meets the condition, then the cumulative number is increased.

[0082] Assume the model fit threshold is Th co (For example, 0.95 for a normal distribution), the model compliance of the first sound signal is greater than or equal to the model compliance threshold Th. co This refers to the model conformity of the first sound signal satisfying...

[0083]

[0084] The cumulative count is an integer greater than or equal to 0, with an initial value of 0. It increases by 1 in each detection cycle. For example, in the first detection cycle after a call begins, if the model compliance of the first audio signal is greater than or equal to the model compliance threshold, and the frequency domain information of the first audio signal meets the condition, the cumulative count increases from 0 to 1. In the second detection cycle after a call begins, if the model compliance of the first audio signal is still greater than or equal to the model compliance threshold, and the frequency domain information of the first audio signal meets the condition, the cumulative count increases from 1 to 2. This continues. If the first microphone is immersed in liquid, and for N consecutive detection cycles the model compliance of the first audio signal is greater than or equal to the model compliance threshold, and the frequency domain information of the first audio signal meets the condition, the cumulative count increases by N.

[0085] S105. In each detection cycle, if the model compliance of the first sound signal is less than the model compliance threshold, or if the frequency domain information of the first sound signal does not meet the conditions, the cumulative number is reduced.

[0086] If a user's finger blocks the first microphone, the blockage will not last. After a period of time, the user's finger will leave the first microphone, and it will no longer be blocked. However, if the first microphone is immersed in liquid, the accumulated count can be reduced if it is not blocked during the current detection cycle. For example, the accumulated count can be reduced by 1 each detection cycle, down to a minimum of 0. The purpose of S105 is to prevent accidental microphone switching due to user blocking.

[0087] After multiple detection cycles, if the cumulative number exceeds the cumulative threshold, step S106 is executed, switching the second microphone to the primary microphone and the first microphone to the secondary microphone. Otherwise, the first microphone remains the primary microphone, and the process jumps back to S101. For example, with a detection cycle of 20ms, if a user takes an average of 400ms to speak a word, approximately 20 detection cycles are required. Assuming the cumulative threshold equals the average time a user takes to speak a word divided by the detection cycle, minus a margin (e.g., 2), the cumulative threshold is 400 / 20 - 2. After 20 detection cycles, if the cumulative number (e.g., 20) exceeds the cumulative threshold (e.g., 18), the second microphone is switched to the primary microphone, and the first microphone is switched to the secondary microphone. In other words, the switching between primary and secondary microphones is determined and completed within the time it takes for the user to speak a word, and the user does not perceive it.

[0088] It should be noted that the reason for requiring multiple testing cycles is to avoid errors caused by interference in a single testing cycle, which could lead to accidental microphone switching.

[0089] S106. Switch the second microphone to the main microphone and the first microphone to the auxiliary microphone.

[0090] The following is combined with Figure 10 The effect of the microphone switching method provided in this application embodiment is illustrated by simulating microphone immersion in liquid by blocking the microphone with a user's finger.

[0091] As shown in A1, assuming the second microphone is blocked by the user's finger, the second microphone will output a sound signal. However, as shown in A2, the sound signal output by the second microphone does not appear in the final acquired sound signal. As shown in B1, assuming the finger blocking the second microphone is removed, the second microphone will still output a sound signal. However, as shown in B2, the sound signal output by the second microphone also does not appear in the final acquired sound signal. This indicates that during this process, the main microphone is always the first microphone, and the auxiliary microphone is always the second microphone.

[0092] As shown in C1, assuming the first microphone is blocked by a user's finger, the first microphone will output a sound signal, and as shown in C2, the sound signal output by the first microphone appears in the final acquired sound signal. As shown in D1, assuming the finger blocking the first microphone is removed, the first microphone will still output a sound signal, but as shown in D2, the sound signal output by the second microphone also does not appear in the final acquired sound signal. Furthermore, the frequency domain information of the final acquired sound signal during C2 to D2 is weaker than that during A2 to B2. After D2, the frequency domain information of the final acquired sound signal returns to the frequency domain information of the sound signal during A2 to B2. This indicates that during C2 to D2, the main microphone switches to the second microphone, and the auxiliary microphone switches to the first microphone. After D2, the main microphone switches back to the first microphone, and the auxiliary microphone switches back to the second microphone.

[0093] The microphone switching method and electronic device provided in this application collect a first sound signal through a first microphone and a second microphone, with the first microphone being the primary microphone. In each detection cycle, if the model compliance of the first sound signal relative to the liquid-immersed microphone model is greater than or equal to a model compliance threshold, and the frequency domain information of the first sound signal meets the conditions, the cumulative count is increased until it exceeds the cumulative threshold. Then, the second microphone is switched to the primary microphone, and the first microphone is switched to the secondary microphone. This achieves microphone switching in a liquid-immersed microphone scenario.

[0094] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A microphone switching method, characterized in that, Applied to an electronic device including a first microphone and a second microphone, wherein the first microphone is a main microphone, the method includes: In each detection cycle, feature extraction is performed on the first sound signal to obtain the frequency domain information of the first sound signal, and feature extraction is performed on the second sound signal to obtain the frequency domain information of the second sound signal; the first sound signal is the sound signal collected through the first microphone, and the second sound signal is the sound signal collected through the second microphone; In each detection cycle, the model compliance of the first sound signal is calculated based on the frequency domain information of the first sound signal and the frequency domain information of the immersion microphone model; the frequency domain information refers to the spectrum or energy; for M first target frequencies, at the i-th first target frequency... Below, the frequency domain information of the first sound signal is The frequency domain information of the liquid-immersed microphone model is The model conformity of the first sound signal is , 1≤i≤M; In each detection cycle, it is determined whether the frequency domain information of the first sound signal meets the conditions; for N second target frequencies, at the j-th second target frequency Below, the frequency domain information of the first sound signal is The frequency domain information of the second sound signal is The frequency domain information of the first sound signal satisfies the condition. ,and, , 1≤j≤N, The difference threshold, The absolute threshold; In each detection cycle, if the model compliance of the first sound signal is greater than or equal to the model compliance threshold, and the frequency domain information of the first sound signal meets the condition, then the cumulative number is increased. After multiple detection cycles, if the cumulative number is greater than the cumulative threshold, the second microphone is switched to the main microphone and the first microphone is switched to the auxiliary microphone.

2. The method according to claim 1, characterized in that, Also includes: If the model compliance of the first sound signal is less than the model compliance threshold, or if the frequency domain information of the first sound signal does not meet the conditions, then the cumulative number is reduced.

3. The method according to claim 1 or 2, characterized in that, In each detection cycle, the model compliance of the first sound signal is calculated based on the frequency domain information of the first sound signal and the frequency domain information of the immersion microphone model, including: If at least a portion of the frequency domain information of the first sound signal is greater than a preset threshold, then in each detection cycle, the model conformity of the first sound signal is calculated based on the frequency domain information of the first sound signal and the frequency domain information of the immersion microphone model.

4. The method according to claim 3, characterized in that, Also includes: If all frequency domain information of the first sound signal is less than a preset threshold, then the second microphone is switched to the main microphone and the first microphone is switched to the auxiliary microphone.

5. An electronic device, characterized in that, The device includes a processor and a memory, wherein the memory stores instructions that, when executed by the processor, cause the electronic device to perform the method as described in any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, The device stores instructions that, when executed on the electronic device, cause the electronic device to perform the method as described in any one of claims 1-4.

7. A computer program product, characterized in that, Includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-4.

Images