Voice wake-up link switching method and electronic equipment

Abstract

The embodiment of the invention provides a voice wake-up link switching method and electronic equipment, and relates to the technical field of intelligent voice control. The method comprises the following steps: acquiring detection audio data obtained by recording a currently played audio by a plurality of microphones, and extracting audio energy characteristics comprising a recording energy value of a single microphone and a recording comparison result of any two microphones from the detection audio data; and based on the audio energy characteristics, a detection model is used to generate switching decision information. And then the switching decision information is written into the voice wake-up control chip, so that the voice wake-up control chip determines a voice wake-up link according to the switching decision information. According to the method, automatic detection and switching of low-power-consumption voice wake-up links can be achieved, when one microphone link of the electronic equipment is abnormal, voice wake-up continues to be conducted through other microphone links, and the voice wake-up rate of the electronic equipment is increased.

Description

Technical Field

[0001] This application relates to the field of intelligent voice control technology, and in particular to a voice wake-up link switching method and electronic device. Background Technology

[0002] Smartphones, tablets, smart learning devices, and other electronic devices can support intelligent voice interaction. Intelligent voice interaction refers to the exchange of information between users and electronic devices through voice input and output. Intelligent voice interaction allows users to interact with electronic devices in natural language, and the electronic devices can understand the user's voice commands and respond with voice or other operational responses.

[0003] For electronic devices that support intelligent voice interaction, the device can be woken up via voice. For example, when the electronic device is in standby mode with the screen off, the user can input a wake-up word by voice. The sound waves from the voice input are picked up by the built-in microphone of the electronic device and converted into electrical signals. Then, acoustic features are extracted from the electrical signals and acoustic model matching is performed. When the confidence score obtained from the model matching is higher than a preset confidence threshold, the main system is activated, thus achieving voice wake-up.

[0004] Because the voice wake-up function of electronic devices is controlled by a low-power voice wake-up control chip, which uses only a single fixed microphone for audio signal acquisition, the voice wake-up control chip struggles to extract high-quality acoustic features from the audio signal when the microphone signal quality is low or malfunctioning. This results in a lower acoustic model matching success rate and consequently reduces the voice wake-up rate of the electronic device. Summary of the Invention

[0005] In view of this, embodiments of this application provide a voice wake-up link switching method and an electronic device to solve the problem of low voice wake-up rate of electronic devices.

[0006] According to a first aspect of this application, a voice wake-up link switching method is provided, the method comprising: Acquire detection audio data, which includes audio signals obtained by recording the currently playing audio through multiple microphones built into the electronic device; Audio energy features are extracted from the detected audio data, including the recording energy value of a single microphone and the recording comparison results of any two microphones; Based on the audio energy characteristics, a detection model is used to generate switching decision information; the detection model is a neural network model trained using sample energy data; the sample energy data includes sample audio collected by multiple microphones and the labeled wake-up rate of the sample audio. The switching decision information is written into the voice wake-up control chip of the electronic device so that the voice wake-up control chip determines the voice wake-up link according to the switching decision information.

[0007] In some embodiments, acquiring detected audio data includes: Monitor audio playback events of electronic devices for target audio, wherein the target audio is a multimedia resource with a playback duration greater than or equal to a preset duration threshold; Upon detecting the audio playback event, receive audio signals recorded by multiple microphones; The detection audio segment is extracted from the audio signal to obtain the detection audio data; the detection audio segment is an audio segment whose audio duration is greater than or equal to the width of a preset analysis window.

[0008] In some embodiments, receiving audio signals recorded by multiple microphones includes: Send a first mode switching command to the voice wake-up control chip so that the voice wake-up control chip enters the detection mode; The voice wake-up control chip sends audio recording commands to multiple microphones. Receive the audio signal recorded by the microphone in response to the audio recording command; When the recording duration of the audio signal exceeds the width of the preset analysis window, a second mode switching command is sent to the voice wake-up control chip to cause the voice wake-up control chip to exit the detection mode.

[0009] In some embodiments, after extracting audio energy features from the detected audio data, the method further includes: Obtain first-order judgment conditions, which include an energy lower limit and an energy relative reduction threshold. The first-order judgment conditions are set based on the normal detection energy of the microphone, or by simulating the microphone volume reduction state in combination with the voice wake-up rate of the electronic device. A first-order judgment result is generated by comparing the audio energy characteristics with the first-order judgment condition; If the first-order judgment result indicates an anomaly, second-order judgment data is generated and input into the detection model; the second-order judgment data includes the audio energy features, the detected audio data, and the device identification information of the electronic device. If the first-order judgment result is that there is no abnormality, a second mode switching command is sent to the voice wake-up control chip to make the voice wake-up control chip exit the detection mode.

[0010] In some embodiments, a first-order judgment result is generated by comparing the audio energy features with the first-order judgment condition, including: Compare the recording energy value of the individual microphone with the lower energy limit; If the recording energy value of a single microphone is lower than the lower energy limit, a first-order judgment result indicating an anomaly is generated; If the recording energy value of a single microphone is higher than the lower energy limit, the recording comparison results of any two microphones are compared with the relative energy reduction threshold to generate the first-order judgment result based on the comparison result; wherein, when the recording comparison result is greater than the relative energy reduction threshold, a first-order judgment result indicating an anomaly is generated; when the recording comparison result is less than or equal to the relative energy reduction threshold, a first-order judgment result indicating no anomaly is generated.

[0011] In some embodiments, generating second-order judgment data and inputting the second-order judgment data into the detection model includes: Obtain the device identification information of the electronic device; Based on the detected audio data, the audio energy characteristics, and the device identification information, second-order judgment data is generated; The second-order judgment data is sent to the server to trigger the server to call the detection model; The second-order judgment data is input into the detection model through the server.

[0012] In some embodiments, based on the audio energy characteristics, a detection model is used to generate switching decision information, including: Extract the device identification information from the second-order judgment data; Based on the device identification information, query the device type and the number of times the electronic device has uploaded data; The second-order judgment data, the device type, and the number of data uploads are input into the detection model to obtain the predicted wake-up rate output by the detection model. Based on the predicted wake-up rate query switching strategy, the switching strategy includes a voice wake-up link that serves as a voice wake-up microphone; The handover decision information is generated based on the handover strategy.

[0013] In some embodiments, the method further includes: Obtain sample energy data; The sample energy data is input into the detection model to obtain the training wake-up rate output by the detection model; The training loss is calculated based on the training wake-up rate and the labeled wake-up rate. The model parameters of the prediction model are adjusted by backpropagating the training loss.

[0014] In some embodiments, the method further includes: Acquire verification audio data, which includes audio signals obtained by recording the wake-up voice through multiple microphones; Extract voice wake-up features from the verification audio data, the voice wake-up features including the audio energy value of a single microphone when the wake-up voice is recorded and the audio comparison results of any two microphones; Wake-up result data is collected based on the voice wake-up features, and the wake-up result data includes the voice wake-up features, the audio signal corresponding to the wake-up voice, and the device identification information of the electronic device. The detection model is iteratively optimized using the wake-up result data.

[0015] According to a second aspect of this application, an electronic device is provided, the electronic device comprising: a processor, a voice wake-up control chip, and a plurality of microphones; the processor being connected to the voice wake-up control chip and the microphones; the processor being configured to: Acquire detection audio data, which includes audio signals obtained by recording the currently playing audio through multiple microphones; Audio energy features are extracted from the detected audio data, including the recording energy value of a single microphone and the recording comparison results of any two microphones; Based on the audio energy characteristics, a detection model is used to generate switching decision information; the detection model is a neural network model trained using sample energy data; the sample energy data includes sample audio collected by multiple microphones and the labeled wake-up rate of the sample audio. The switching decision information is written into the voice wake-up control chip so that the voice wake-up control chip determines the voice wake-up link according to the switching decision information.

[0016] According to a third aspect of this application, a voice wake-up link switching device is provided, the device comprising: An audio acquisition module is used to acquire detected audio data, which includes audio signals obtained by recording the currently playing audio through multiple microphones built into the electronic device. The feature extraction module is used to extract audio energy features from the detected audio data, the audio energy features including the recording energy value of a single microphone and the recording comparison results of any two microphones; A switching decision module is used to generate switching decision information based on the audio energy features using a detection model; the detection model is a neural network model trained using sample energy data; the sample energy data includes sample audio collected by multiple microphones and the labeled wake-up rate of the sample audio. The strategy writing module is used to write the switching decision information into the voice wake-up control chip of the electronic device, so that the voice wake-up control chip determines the voice wake-up link according to the switching decision information.

[0017] According to a fourth aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described voice wake-up link switching method.

[0018] By employing the above technical solution, embodiments of this application provide a voice wake-up link switching method and an electronic device. The method acquires detection audio data obtained by recording currently playing audio using multiple microphones, and extracts audio energy features from the detection audio data, including the recording energy value of a single microphone and the comparison results of recordings from any two microphones. Based on the audio energy features, a detection model is used to generate switching decision information. This switching decision information is then written to a voice wake-up control chip, enabling the chip to determine the voice wake-up link according to the switching decision information. This method can achieve automatic detection and switching of low-power voice wake-up links. When one microphone link of the electronic device malfunctions, voice wake-up continues through other microphone links, improving the voice wake-up rate of the electronic device. Furthermore, the switching process can be performed in the background, achieving seamless switching for the user.

[0019] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the voice wake-up process provided in an embodiment of this application; Figure 2 This is a schematic flowchart of the voice wake-up link switching method provided in the embodiments of this application; Figure 3 This is a schematic diagram of a first-order judgment process provided in an embodiment of this application; Figure 4 This is a schematic diagram of the anomaly reporting process provided in an embodiment of this application; Figure 5 This is a schematic diagram of a second-order judgment process provided in an embodiment of this application; Figure 6 This is a schematic diagram of the iterative optimization process of the detection model provided in the embodiments of this application; Figure 7 This is a schematic diagram of the voice wake-up link switching device provided in an embodiment of this application. Detailed Implementation

[0021] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0022] In this embodiment, the voice wake-up link switching method can be used by electronic devices to perform intelligent voice interaction. Intelligent voice interaction refers to a technology that enables information exchange between a user and an electronic device through voice input and output. Intelligent voice interaction allows users to interact with electronic devices in natural language, and the electronic device can understand the user's voice commands and respond with voice or other operational responses.

[0023] Taking smartphones, tablets, and smart learning machines—electronic devices that support intelligent voice interaction—as examples, these devices can be woken up through voice interaction. Voice wake-up refers to the user's ability to input a wake-up word when the electronic device is in a low-power state, such as screen-off standby, to trigger the device to exit the low-power state and enter normal use mode.

[0024] like Figure 1 As shown, in some embodiments, to achieve voice wake-up, the electronic device can detect sound through a built-in microphone (MIC) and convert the sound waves of the voice input into electrical signals that can be processed by the control chip. For example, after the sound waves are converted into analog electrical signals by the microphone, they are then sampled into digital waveform audio signals by an analog-to-digital converter (ADC) according to preset sampling parameters, such as a sampling rate of 16kHz and a precision of 16bit.

[0025] During audio signal acquisition, preprocessing can be performed on the sampled signal. For example, audio signal preprocessing can include denoising, Voice Activity Detection (VAD), and gain control. Denoising refers to removing environmental noise using techniques such as spectral subtraction or Wiener filtering. VAD detects whether the input signal is speech, enabling subsequent processing to begin when speech is present, thus reducing power consumption. Gain control automatically adjusts the signal amplitude to prevent feature distortion caused by excessively strong or weak signals.

[0026] Then, acoustic features are extracted based on the preprocessed audio signal. For example, the processed audio signal is converted into a set of condensed digital fingerprints by using Mel-Frequency Cepstral Coefficients (MFCC) or Mel Spectrogram methods. That is, by performing a nonlinear transformation on the frequency of the sound, a small segment of audio is converted into a vector representing the sound feature.

[0027] After obtaining sound feature vectors through acoustic feature extraction, acoustic model matching can be performed based on these feature vectors. For example, deep learning models such as Deep Neural Networks (DNNs), Convolutional Neural Networks (CNNs), and Recurrent Neural Networks (RNNs) can be used to analyze the extracted sound feature vectors, determine whether the sound feature vectors match a preset wake word, and output a confidence score.

[0028] After obtaining the confidence score output by the model, the electronic device can make a confidence decision based on the confidence score and a preset confidence threshold. That is, when the confidence score is higher than the preset confidence threshold, it is determined as "wake-up successful", and thus the main system can be activated to achieve voice wake-up.

[0029] To enable continuous monitoring and voice wake-up, electronic devices can incorporate a voice wake-up control chip. This chip is a dedicated control device for the voice wake-up process, characterized by ultra-low power consumption. For example, a voice wake-up control chip can be a Digital Signal Processor (DSP) or a Neural Processing Unit (NPU) to achieve 24 / 7 uninterrupted voice monitoring while conserving power.

[0030] Because the voice wake-up function of electronic devices is controlled by a low-power voice wake-up control chip, which uses only a single fixed microphone for audio signal acquisition, the voice wake-up control chip struggles to extract high-quality acoustic features from the audio signal when the microphone signal quality is low or malfunctioning. This results in a lower acoustic model matching success rate and consequently reduces the voice wake-up rate of the electronic device.

[0031] For example, smartphones, tablets, and smart learning devices typically use only one microphone (MIC) for audio signal acquisition during low-power wake-up. If this MIC malfunctions, accurate voice wake-up will fail, requiring microphone hardware replacement and resulting in a decreased user experience. Furthermore, since some electronic devices have multiple built-in microphones, when one is used as the wake-up microphone, the others remain idle and cannot serve the user's low-power voice wake-up needs.

[0032] To address the issue of low voice wake-up rates in electronic devices, this application provides a voice wake-up link switching method in some embodiments. This method enables automatic detection and switching of MIC links in low-power voice wake-up scenarios, ensuring that if one MIC link fails, another MIC link can continue to be used. This reduces the impact of microphone link abnormalities on the voice wake-up process, improves the voice wake-up rate of electronic devices, and enhances the user experience.

[0033] The method can be applied to electronic devices with data processing capabilities. These electronic devices include, but are not limited to, computers, servers, mobile terminals, smart learning machines, smart wearable devices, and industrial control machines. For ease of description, this application embodiment uses an electronic device as the execution subject of the method. It should be understood that the method can also be applied to other types of execution subjects, which are not illustrated in this application embodiment. Figure 2 As shown, the method includes: S101. Obtain the detected audio data.

[0034] When switching voice wake-up links, the electronic device can detect the sound it is playing to obtain detection audio data. This detection audio data includes audio signals obtained by recording the currently playing audio using multiple microphones built into the electronic device. It should be noted that the multiple microphones include at least the microphone currently used for voice wake-up detection.

[0035] For example, when an electronic device has two built-in microphones, MIC1 and MIC2, and the electronic device is currently using MIC1 for voice wake-up detection, the audio signals detected by both microphones, MIC1 and MIC2, can be acquired simultaneously during audio playback. That is, the detected audio data includes two audio signals, AS1 obtained by MIC1 recording the played audio and AS2 obtained by MIC2 recording the played audio.

[0036] In some embodiments, when acquiring detection audio data, it is possible to first detect whether the electronic device is playing audio and whether the played audio meets the requirements for voice wake-up link switching judgment. That is, the audio playback event of the electronic device for the target audio can be monitored. The target audio is a multimedia resource with a playback duration greater than or equal to a preset duration threshold.

[0037] For example, event listeners can be added to the operating system of an electronic device or to audio monitoring applications. Event listeners can monitor all operational events on an electronic device. When an event listener detects an audio playback (AP) event, it can detect the audio being played to determine if the audio playback event is indeed the playback event for the target audio.

[0038] To facilitate sound recording, the target audio to be played should have a certain playback duration, and there should be recordable audio content during the audio playback process. That is, the target audio is non-silent audio or non-very quiet audio with a certain playback duration.

[0039] For example, if the preset duration threshold is 10 seconds, when an audio playback event is detected, the corresponding audio information can be extracted from the event, and the total duration of the played audio can be determined. If the total duration exceeds 10 seconds, and the volume (or amplitude) of the played audio exceeds the preset minimum volume value (or minimum amplitude), the played audio is determined to be the target audio. Accordingly, the audio playback event is designated as an audio playback event for the target audio.

[0040] When an audio playback event is detected, audio signals recorded by multiple microphones can be received. That is, when the electronic device is detected playing target audio, the sound of the playing audio is recorded by multiple microphones built into the electronic device to obtain audio signals recorded by multiple microphones.

[0041] In some embodiments, the audio recording process of multiple microphones can be controlled by a voice wake-up control chip. Specifically, when receiving audio signals recorded by multiple microphones, a first mode switching command can be sent to the voice wake-up control chip to put it into detection mode. The voice wake-up control chip then sends audio recording commands to the multiple microphones and receives the audio signals recorded by the microphones in response to the recording commands.

[0042] During the process of receiving audio signals, the recording duration of the received audio signals can also be recorded. When the recording duration of the audio signals exceeds the width of the preset analysis window, a second mode switching command is sent to the voice wake-up control chip so that the voice wake-up control chip exits the detection mode.

[0043] For example, an electronic device can detect audio playback events through an audio playback (AP) application. When the AP detects that the electronic device is playing target audio, it can activate a low-power voice wake-up control chip via a first mode switching command (such as a detection notification), causing the voice wake-up control chip to enter detection mode. In detection mode, the low-power voice wake-up control chip can control multiple microphones built into the electronic device to record audio via audio recording commands, and acquire the audio signals recorded by the microphones, thus achieving audio signal collection. When the duration of the recorded audio signal exceeds the width of a preset analysis window (such as 5 seconds), it can be determined that the collection is complete. After the audio signal collection is complete, a second mode switching command (such as an interrupt notification) causes the voice wake-up control chip to exit detection mode, thereby stopping the acquisition of audio signals.

[0044] After receiving audio signals recorded by multiple microphones, audio segments can be extracted from the audio signals to obtain detection audio segments suitable for energy analysis. These detection audio segments are audio segments with an audio duration greater than or equal to the width of a preset analysis window. Therefore, the electronic device can extract multiple detection audio segments from the audio signals recorded by multiple microphones at the same extraction time point to obtain detection audio data.

[0045] In some embodiments, in order to extract a suitable detection audio segment from the audio signal for energy analysis, the corresponding audio file of the target audio can be obtained during audio segment extraction. Waveform extraction is then performed on the audio file to obtain an audio waveform image. The audio waveform image is a visual representation of the audio signal in the time domain.

[0046] Next, sound density analysis is performed on the audio waveform image to obtain dense sound regions. Then, segment truncation points are set based on the dense sound regions and a preset analysis window width. These segment truncation points can include a start point and an end point, and the time interval between the start and end points is greater than or equal to the preset analysis window width. Based on these segment truncation points, detection audio segments are extracted from multiple audio signals.

[0047] For example, if the preset analysis window width is 5 seconds, when extracting an audio segment, the audio file corresponding to the target audio, i.e., audio A, can be obtained first. Then, an audio waveform image can be generated based on the audio file corresponding to audio A using an audio editing application. Next, an image processing model can be used to detect the amplitude values ​​at each time point from the audio waveform image, and then the sound density can be determined based on the amplitude values. For example, sound density is the ratio of the sum of the absolute values ​​of multiple discrete amplitude values ​​to time.

[0048] Based on the calculated sound density, the 5-second file segment with the highest sound density in the audio file is identified, thereby determining the starting point t corresponding to that file segment. s and the endpoint t t That is, from the audio file t s To t t The sound density is highest within a specific time period, therefore, by converting the time corresponding to the file segment to world time, it can be determined based on the starting point t. s Determine the starting point of the interception, based on the ending point t. t Determine the endpoint of the segmentation. Extract audio segments from the audio signals recorded by multiple microphones within the time period from the starting point to the endpoint of the segmentation, thereby obtaining multiple detection audio segments and forming the detection audio data.

[0049] S102. Extract audio energy features from the detected audio data.

[0050] After acquiring the detected audio data, the electronic device can extract audio energy features from the audio data. These audio energy features include the recording energy value of a single microphone and a comparison of recordings from any two microphones.

[0051] For example, the energy value of an audio signal is the sum of the squares of the signal amplitudes. Therefore, when extracting audio energy features, we can first read the amplitude values ​​corresponding to each sampling time point from the audio signal. x [ n Then, by calculating the sum of squares of the signal amplitudes, the energy value of the audio signal is obtained, i.e.:

[0052] in, E This represents the energy value of the audio signal; N This indicates the total number of sampling points in the audio signal; x [ n ] indicates that the audio signal is at the 1st digit. n The amplitude value of each sampling point.

[0053] Since the detected audio data may include multiple audio signals recorded by multiple microphones, the recorded energy values ​​corresponding to multiple audio signals can be obtained by calculating the energy value of each audio signal. E ={ E i , i =1, 2, ..., m},in, E i Indicates the first i The energy value of the audio signal recorded by each microphone, that is, the recording energy value of a single microphone; m This indicates the total number of microphones built into the electronic device.

[0054] Calculate the recording energy value of a single microphone. E i Subsequently, the electronic device can also perform comparative analysis based on the recording energy values ​​of a single microphone to obtain the recording comparison results of any two microphones. The recording comparison results can include the difference in energy values ​​between the two microphones. Ec ij =| E i -E j | .in, E i Indicates the first i The energy value of the audio signal recorded by each microphone; E j Indicates the first j The energy value of the audio signal recorded by each microphone; Ec ij Indicates the first i The microphone and the first j Comparison of recording results from each microphone.

[0055] In some embodiments, to accurately reflect the microphone's sound recording effect, the sound pressure level can be calculated based on the recorded energy value during the extraction of audio energy features. The unit of sound pressure level is decibel (dB). As a physical quantity measuring sound intensity, sound pressure level correlates the energy value of a digital audio signal with physical energy that reflects the recording effect.

[0056] For example, in calculating the recording energy value of a single microphone. E i Then, the sound pressure level (SPL) can be calculated using the following formula:

[0057] Where SPL represents sound pressure level; P0 represents reference sound pressure, such as 20 μPa; and P represents sound pressure, which is related to energy value. E Physical quantities that have mathematical relationships, i.e. E = P 2 / 2 ρc .in, ρ It is the density of air, which is 1.21 kg / m³. 3 ; c That is the speed of sound, which is 343 m / s.

[0058] By calculating the sound pressure level, the sound pressure level can be used to represent the recorded energy value corresponding to the audio signal. The energy value of a single microphone is then expressed as: V iThe energy difference between the two MICs is expressed as ΔV ij .

[0059] After extracting audio energy features from the audio data, the health of the MIC link can be measured based on the extracted audio energy features. The measurement results of the health of the corresponding link of each MIC can be used to determine whether the current voice wake-up link is in normal use.

[0060] like Figure 3 As shown, in some embodiments, after extracting audio energy features from the detected audio data, a first-order judgment can be performed first. For example, based on the extracted audio energy features, a judgment can be performed locally on the electronic device. When performing the first-order judgment, first-order judgment conditions can be obtained first. These first-order judgment conditions include an energy lower limit V. limit And the energy relative decrease threshold V threshold .

[0061] The first-order judgment condition can be set based on the microphone's normal detection energy. For example, the lower energy limit V. limit It can be set to be 12dB (or lower) lower than the normal acquisition value of the MIC, and the specific value depends on the acceptable value range of different algorithms and electronic devices. This is in contrast to relatively lowering the threshold V. threshold The value can be compared with the lower energy limit V. limit To maintain consistency, the value can also be set based on the energy difference between multiple microphones during normal acquisition. The specific value depends on the range of values ​​that different algorithms and electronic devices can accept.

[0062] First-order judgment conditions can also be combined with the simulated microphone volume reduction state and the voice wake-up rate setting of the electronic device. For example, MIC audio signals at different energy values ​​can be simulated and input into an acoustic model. Then, the confidence scores of the acoustic model at different energy values ​​can be counted, and the number of times the electronic device is woken up by voice can be determined based on the confidence scores and a preset confidence threshold, thereby calculating the wake-up rate. The wake-up rate can then be combined to select which value is more reasonable, namely, the lowest energy value and the energy difference at which the confidence score reaches the preset confidence threshold, which serve as the lower limit of energy and the relative reduction threshold.

[0063] After obtaining the first-order judgment conditions, a first-order judgment result can be generated by comparing the audio energy characteristics with the first-order judgment conditions. Specifically, the first-order judgment result can be generated by comparing the recording energy value of a single microphone with the lower limit of energy, and by comparing the recording comparison results of any two microphones with the relative energy reduction threshold.

[0064] The electronic device can first compare the recording energy value of a single microphone with the lower energy limit. If the recording energy value of a single microphone is lower than the lower energy limit, a first-order judgment result indicating an anomaly is generated. For example, the electronic device determines that the recording energy value of a microphone is lower than V. limit If the MIC is found to be abnormal, no further testing is needed, and a first-order judgment result indicating the presence of an abnormality can be generated directly.

[0065] If the recording energy value of a single microphone is higher than the lower energy limit, it is necessary to further compare the recording comparison results of any two microphones with the relative energy reduction threshold to generate a first-order judgment result based on the comparison result. Specifically, if the recording comparison result is greater than the relative energy reduction threshold, a first-order judgment result indicating an anomaly is generated; if the recording comparison result is less than or equal to the relative energy reduction threshold, a first-order judgment result indicating no anomaly is generated.

[0066] For example, if the recording energy value of the MIC is higher than V limit The electronic device then compares the energy values ​​between the two microphones. Through this comparison, it determines whether the difference between the energy value of the current microphone and the energy value of the other microphone is higher than the relative reduction threshold V. threshold If the difference in recording energy values ​​between the audio signals recorded by the two microphones is too large, it can be determined that at least one of the two microphones has a detection anomaly, and thus a first-order judgment result indicating an anomaly can be generated.

[0067] Based on the first-order judgment result, if the first-order judgment result is that there is no abnormality, it means that the current voice wake-up link is not abnormal. Therefore, a second mode switching command can be sent to the voice wake-up control chip to make the voice wake-up control chip exit the detection mode.

[0068] If the first-order judgment indicates an anomaly, a second-order judgment can be performed. This means the electronic device can generate second-order judgment data and input it into the detection model. The second-order judgment data includes audio energy characteristics, detected audio data, and the electronic device's identification information.

[0069] To achieve second-order judgment, in some embodiments, the electronic device can acquire its device identification information and generate second-order judgment data based on the detected audio data, audio energy characteristics, and device identification information. This second-order judgment data is then sent to the server to trigger the server to invoke the detection model, thereby inputting the second-order judgment data into the detection model through the server.

[0070] For example, such as Figure 4As shown, after an electronic device detects an anomaly in the voice wake-up link through a low-power voice wake-up control chip, it can notify the main CPU of the electronic device, namely the microcontroller unit (MCU), via interrupt and inter-integrated circuit (I2C) communication. The MCU then uploads the data to the server via the network. The second-order decision data uploaded to the server includes, but is not limited to, the detected energy value, the energy difference between the two microphones, the audio signal (detected audio data), and the device type, so that the server can make a decision using the detection model.

[0071] S103. Based on audio energy characteristics, use a detection model to generate switching decision information.

[0072] After acquiring audio energy features, a detection model can be used to generate switching decision information. This detection model is a type of decision model that can be used to predict wake-up rates based on audio energy features and generate switching decision information based on the predicted wake-up rates.

[0073] In some embodiments, the detection model is a neural network model trained using sample energy data. The sample energy data includes sample audio collected from multiple microphones and the labeled wake-up rates of the sample audio. Therefore, before performing a voice wake-up link switch, the detection model needs to be trained based on the sample energy data. That is, by acquiring the sample energy data and inputting it into the detection model, the training wake-up rate output by the detection model is obtained. Then, the training loss is calculated based on the training wake-up rate and the labeled wake-up rate, and the model parameters of the prediction model are adjusted through backpropagation of the training loss.

[0074] For example, when training a detection model, sample energy data can be acquired first. Audio samples are extracted from the audio dataset through data collection and preprocessed, such as through noise reduction and normalization, to ensure data quality. Then, the energy value of each audio sample is calculated. Finally, each sample is labeled with its wake-up rate.

[0075] The calculated sample energy data is then used as input features into the detection model. The detection model can function as a deep learning neural model, predicting results based on initialized model parameters. In other words, the detection model calculates based on the input energy data and outputs the predicted wake-up rate for each sample.

[0076] Next, the training loss is calculated based on the training and labeled wake-up rates. An appropriate loss function is chosen to measure the difference between the model's predicted values ​​and the true labeled values. Loss functions include mean squared error (MSE) and cross-entropy loss. Then, for each sample, the loss is calculated by comparing the training wake-up rate output by the detection model with the labeled wake-up rate. The loss values ​​for all samples are summed to obtain the total loss, which is used to evaluate the overall performance of the detection model.

[0077] Then, based on the calculated total loss, the gradient of the loss with respect to the model parameters is calculated using the backpropagation algorithm. Backpropagation can start from the output layer and calculate the gradient layer by layer until it reaches the input layer. Optimization algorithms such as gradient descent and Adam are used to update the model parameters based on the gradient. Through iterative training, the model parameters of the detection model are continuously adjusted until the loss value of the detection model converges to a low value, or until a preset number of training epochs are reached, thus obtaining an audio detection model that can effectively predict wake-up rates.

[0078] The detection model can run locally on the electronic device, meaning it can generate handover decision information based on audio energy features at the local end. Alternatively, the detection model can run on a server connected to the electronic device, meaning it can generate handover decision information based on audio energy features at the server end.

[0079] Taking the local device as an example, electronic devices can use audio energy features as input data to the detection model. The detection model can then predict the wake-up rate based on the audio energy features, that is, use a trained neural network to calculate the predicted classification probability of the audio energy features against the wake-up rate label.

[0080] After obtaining the predicted wake-up rate, the electronic device compares the predicted wake-up rate with a pre-set wake-up rate threshold. When the predicted wake-up rate is higher than the threshold, it is determined that the current voice wake-up link can perform voice wake-up normally, thus generating switching decision information that does not require link switching. Conversely, when the predicted wake-up rate is lower than the threshold, it is determined that the current voice wake-up link cannot meet the voice wake-up operation, and therefore switching decision information that requires link switching can be generated.

[0081] When generating decision information, the detection model can refer to other information in addition to audio energy features to achieve more accurate anomaly detection in the voice wake-up link. The referenced information can include the detected audio data and the device identification information of the electronic device. Specifically, the device identification information can be used to determine the device type and the number of times the electronic device has uploaded data.

[0082] Therefore, in some embodiments, when generating switching decision information using a detection model based on audio energy features, device identification information can be extracted from the second-order judgment data. Then, the device type and the number of data uploads by the electronic device are queried based on the device identification information. The second-order judgment data, device type, and number of data uploads are then input into the detection model to obtain the predicted wake-up rate output by the detection model. A switching strategy is then queried based on the predicted wake-up rate, and switching decision information is generated according to the switching strategy. The switching strategy includes a voice wake-up link that serves as the voice wake-up microphone.

[0083] For example, such as Figure 5 As shown, when the server generates a handover decision, configuration data such as the handover strategy, threshold, and detection model can be imported into the server first, enabling the server to form a second-order judgment module for handover decision generation. After the electronic device's MCU reports information such as energy value, the difference between the two microphones, detected audio data, and device identification information to the server via the network, the server can perform second-order judgments based on the detection model, considering data, energy value, and upload count, to determine if there is an anomaly in the current electronic device's voice wake-up link. If an anomaly is detected, a strategy query can be performed based on the predicted wake-up rate to obtain the handover strategy. Then, handover decision information is generated based on the handover strategy and sent to the electronic device.

[0084] S104. Write the switching decision information into the voice wake-up control chip of the electronic device so that the voice wake-up control chip determines the voice wake-up link according to the switching decision information.

[0085] After generating handover decision information using the detection model, the electronic device can write this information into the voice wake-up control chip, enabling the chip to determine the voice wake-up link based on this information. The handover decision information can include two scenarios: whether the voice wake-up link needs to be switched or not.

[0086] In cases where there is no need to switch the voice wake-up link, the electronic device does not need to rewrite the switching decision information into the voice wake-up control chip. Instead, it can send a second mode switching command to the voice wake-up control chip to make the voice wake-up control chip exit the detection mode and collect and judge the voice wake-up audio signal according to the previous voice wake-up link.

[0087] In cases where a switch of the voice wake-up link is required, the electronic device can transmit the switching decision information to the voice wake-up control chip. After receiving the switching decision information, the voice wake-up control chip reads the switched voice wake-up link from the switching decision information and sets the switched voice wake-up link to the voice wake-up control chip. This allows the electronic device to collect voice wake-up signals through the switched voice wake-up link in the subsequent voice wake-up process.

[0088] In some embodiments, the electronic device reads switching decision information from the MCU to determine the new voice wake-up link. Then, it generates a chip configuration program based on the new voice wake-up link and sends it to the voice wake-up control chip. This allows the voice wake-up control chip to replace the original voice wake-up control program with the new chip configuration program, thus achieving the voice wake-up link switching.

[0089] By applying the technical solutions of the above embodiments, the voice wake-up link switching method described in the above embodiments can introduce a detection model to perform second-order judgment for MIC anomalies, thereby realizing voice wake-up link switching. The voice wake-up link switching result is also bound to the electronic device, enabling each electronic device to adaptively select the voice wake-up link based on different underlying hardware path states, thus improving the success rate of the voice wake-up process.

[0090] In some embodiments, as a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process of this embodiment, some embodiments of this application also provide a voice wake-up link switching method. The difference between this method and the above embodiments is that the electronic device can collect the results by returning the user's usage results after the initial wake-up and switching, so as to achieve continuous monitoring of the voice wake-up link switching results. Figure 6 As shown, the method includes: S201. Obtain verification audio data; S202. Extract voice wake-up features from the verification audio data; S203. Collect wake-up result data based on voice wake-up characteristics; S204. Use the wake-up result data to iteratively optimize the detection model.

[0091] To collect results, after writing the switching decision information into the voice wake-up control chip of the electronic device, verification audio data can be obtained. This verification audio data includes audio signals obtained by recording the wake-up voice through multiple microphones. Similarly, the multiple microphones corresponding to the verification audio data include the microphone corresponding to the voice wake-up link after the switch.

[0092] Then, using the same feature extraction method for audio energy characteristics, voice wake-up features are extracted from the verification audio data. Therefore, the voice wake-up features include the audio energy value of a single microphone when the wake-up voice is recorded and the audio comparison results of any two microphones.

[0093] Then, wake-up result data is collected based on voice wake-up features, including voice wake-up features, the audio signal corresponding to the wake-up voice, and the device identification information of the electronic device. The detection model is then iteratively optimized using this wake-up result data.

[0094] For example, audio can be recorded using multiple microphones built into an electronic device, and the recorded audio signals can be analyzed for content. If any microphone's recorded audio signal contains an audio segment with wake-up word characteristics, it can be determined that the audio signal was obtained by recording the wake-up voice. In this case, the electronic device can obtain audio signals recorded by multiple microphones within the same time period, forming verification audio data. Then, by calculating the audio energy value of a single microphone and comparing the audio results of any two microphones, voice wake-up features are extracted from the verification audio data. Wake-up result data is then collected based on these features, including the voice wake-up features, the audio signal corresponding to the wake-up voice, and the device identification information of the electronic device. The number of times the electronic device is woken up during multiple interactions is then counted to obtain the verification wake-up rate.

[0095] Then, using the verification wake-up rate as the predicted label and voice wake-up features as sample data, the detection model is iteratively trained. The specific process of iterative training is similar to the detection model training process described in the previous embodiment, except that the training loss is calculated based on the predicted wake-up rate and the verification wake-up rate. Through multiple iterations of training, the detection model can be continuously optimized, enabling iterative upgrades to the detection logic, detection threshold, and model parameters, thereby improving the output accuracy of the detection model in subsequent judgment processes.

[0096] In some embodiments, as a specific implementation of the voice wake-up link switching method described in the above embodiments, some embodiments of this application also provide a voice wake-up link switching device, such as... Figure 7 As shown, the device includes: An audio acquisition module is used to acquire detected audio data, which includes audio signals obtained by recording the currently playing audio through multiple microphones built into the electronic device. The feature extraction module is used to extract audio energy features from the detected audio data, the audio energy features including the recording energy value of a single microphone and the recording comparison results of any two microphones; A switching decision module is used to generate switching decision information based on the audio energy features using a detection model; the detection model is a neural network model trained using sample energy data; the sample energy data includes sample audio collected by multiple microphones and the labeled wake-up rate of the sample audio. The strategy writing module is used to write the switching decision information into the voice wake-up control chip of the electronic device, so that the voice wake-up control chip determines the voice wake-up link according to the switching decision information.

[0097] It should be noted that other corresponding descriptions of the functional units involved in the voice wake-up link switching device provided in the embodiments of this application can be found in the corresponding descriptions in the voice wake-up link switching method provided in the above embodiments, and will not be repeated here.

[0098] By applying the technical solutions of the above embodiments, the voice wake-up link switching device described in the above embodiments can acquire detection audio data obtained by recording the currently playing audio from multiple microphones through the audio acquisition module, and extract audio energy features from the detection audio data, including the recording energy value of a single microphone and the recording comparison results of any two microphones, through the feature extraction module. Then, the switching decision module generates switching decision information based on the audio energy features using a detection model. Finally, the switching decision information is written to the voice wake-up control chip through the strategy writing module, so that the voice wake-up control chip determines the voice wake-up link according to the switching decision information. The device can achieve automatic detection and switching of low-power voice wake-up links. When one microphone link of the electronic device malfunctions, voice wake-up continues through other microphone links, improving the voice wake-up rate of the electronic device. Furthermore, the switching process can be operated in the background, achieving seamless switching for the user.

[0099] In some embodiments, as a specific implementation of the voice wake-up link switching method described in the above embodiments, some embodiments of this application also provide an electronic device, which includes a processor, a voice wake-up control chip, and a plurality of microphones; the processor is connected to the voice wake-up control chip and the microphones; the processor is configured to execute the voice wake-up link switching method described in the above embodiments.

[0100] The electronic device may further include a bus, memory, communication interface, input / output interface, and display. The processor of the electronic device provides computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the electronic device stores location information. The network interface of the electronic device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0101] Those skilled in the art will understand that the structure of the electronic device described above is only a partial structure related to the solution of this application and does not constitute a limitation on the electronic device to which the solution of this application is applied. A specific electronic device may include more or fewer components, or combine certain components, or have different component arrangements.

[0102] In one embodiment, a computer-readable storage medium is also provided, which may be non-volatile or volatile, and a computer program is stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0103] In one embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0104] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0105] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0106] Any references to memory, database, or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc.

[0107] Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take many forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0108] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchain. The processors involved in the embodiments provided in this application may be, but are not limited to, general-purpose processors, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc.

[0109] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A voice wake-up link switching method, characterized in that, The method includes: Acquire detection audio data, which includes audio signals obtained by recording the currently playing audio through multiple microphones built into the electronic device; Audio energy features are extracted from the detected audio data, including the recording energy value of a single microphone and the recording comparison results of any two microphones; Based on the audio energy characteristics, a detection model is used to generate switching decision information; the detection model is a neural network model trained using sample energy data; the sample energy data includes sample audio collected by multiple microphones and the labeled wake-up rate of the sample audio. The switching decision information is written into the voice wake-up control chip of the electronic device so that the voice wake-up control chip determines the voice wake-up link according to the switching decision information.

2. The method according to claim 1, characterized in that, Acquire detection audio data, including: Monitor audio playback events of electronic devices for target audio, wherein the target audio is a multimedia resource with a playback duration greater than or equal to a preset duration threshold; Upon detecting the audio playback event, receive audio signals recorded by multiple microphones; The detection audio segment is extracted from the audio signal to obtain the detection audio data; the detection audio segment is an audio segment whose audio duration is greater than or equal to the width of a preset analysis window.

3. The method according to claim 2, characterized in that, Receives audio signals recorded by multiple microphones, including: Send a first mode switching command to the voice wake-up control chip so that the voice wake-up control chip enters the detection mode; The voice wake-up control chip sends audio recording commands to multiple microphones. Receive the audio signal recorded by the microphone in response to the audio recording command; When the recording duration of the audio signal exceeds the width of the preset analysis window, a second mode switching command is sent to the voice wake-up control chip to cause the voice wake-up control chip to exit the detection mode.

4. The method according to claim 1, characterized in that, After extracting audio energy features from the detected audio data, the method further includes: Obtain first-order judgment conditions, which include an energy lower limit and an energy relative reduction threshold. The first-order judgment conditions are set based on the normal detection energy of the microphone, or by simulating the microphone volume reduction state in combination with the voice wake-up rate of the electronic device. A first-order judgment result is generated by comparing the audio energy characteristics with the first-order judgment condition; If the first-order judgment result indicates an anomaly, second-order judgment data is generated and input into the detection model; the second-order judgment data includes the audio energy features, the detected audio data, and the device identification information of the electronic device. If the first-order judgment result is that there is no abnormality, a second mode switching command is sent to the voice wake-up control chip to make the voice wake-up control chip exit the detection mode.

5. The method according to claim 4, characterized in that, By comparing the audio energy features with the first-order judgment condition, a first-order judgment result is generated, including: Compare the recording energy value of the individual microphone with the lower energy limit; If the recording energy value of a single microphone is lower than the lower energy limit, a first-order judgment result indicating an anomaly is generated; If the recording energy value of a single microphone is higher than the lower energy limit, the recording comparison results of any two microphones are compared with the relative energy reduction threshold to generate the first-order judgment result based on the comparison result; wherein, when the recording comparison result is greater than the relative energy reduction threshold, a first-order judgment result indicating an anomaly is generated; when the recording comparison result is less than or equal to the relative energy reduction threshold, a first-order judgment result indicating no anomaly is generated.

6. The method according to claim 4, characterized in that, Generating second-order decision data and inputting the second-order decision data into the detection model includes: Obtain the device identification information of the electronic device; Based on the detected audio data, the audio energy characteristics, and the device identification information, second-order judgment data is generated; The second-order judgment data is sent to the server to trigger the server to call the detection model; The second-order judgment data is input into the detection model through the server.

7. The method according to claim 4, characterized in that, Based on the aforementioned audio energy characteristics, a detection model is used to generate switching decision information, including: Extract the device identification information from the second-order judgment data; Based on the device identification information, query the device type and the number of times the electronic device has uploaded data; The second-order judgment data, the device type, and the number of data uploads are input into the detection model to obtain the predicted wake-up rate output by the detection model. Based on the predicted wake-up rate query switching strategy, the switching strategy includes a voice wake-up link that serves as a voice wake-up microphone; The handover decision information is generated based on the handover strategy.

8. The method according to claim 1, characterized in that, The method further includes: Obtain sample energy data; The sample energy data is input into the detection model to obtain the training wake-up rate output by the detection model; The training loss is calculated based on the training wake-up rate and the labeled wake-up rate. The model parameters of the prediction model are adjusted by backpropagating the training loss.

9. The method according to claim 1, characterized in that, The method further includes: Acquire verification audio data, which includes audio signals obtained by recording the wake-up voice through multiple microphones; Extract voice wake-up features from the verification audio data, the voice wake-up features including the audio energy value of a single microphone when the wake-up voice is recorded and the audio comparison results of any two microphones; Wake-up result data is collected based on the voice wake-up features, and the wake-up result data includes the voice wake-up features, the audio signal corresponding to the wake-up voice, and the device identification information of the electronic device. The detection model is iteratively optimized using the wake-up result data.

10. An electronic device, characterized in that, The electronic device includes: a processor, a voice wake-up control chip, and multiple microphones; the processor is connected to the voice wake-up control chip and the microphones; the processor is configured to: Acquire detection audio data, which includes audio signals obtained by recording the currently playing audio through multiple microphones; Audio energy features are extracted from the detected audio data, including the recording energy value of a single microphone and the recording comparison results of any two microphones; Based on the audio energy characteristics, a detection model is used to generate switching decision information; the detection model is a neural network model trained using sample energy data; the sample energy data includes sample audio collected by multiple microphones and the labeled wake-up rate of the sample audio. The switching decision information is written into the voice wake-up control chip so that the voice wake-up control chip determines the voice wake-up link according to the switching decision information.

Images