Intelligent voice assistant awakening method based on bone conduction

Abstract

The invention provides an intelligent voice assistant awakening method based on bone conduction. The method comprises the following steps: acquiring a bone conduction signal; acquiring a bone conduction signal; determining that the bone conduction signal is a voice segment, and obtaining an original sound signal; performing feature extraction on the original sound signal to obtain a sound feature array; and calculating a primary posterior probability based on the sound feature array, if the primary posterior probability is greater than or equal to a preset primary wake-up threshold, calculating a secondary posterior probability based on the bone conduction signal, and if the secondary posterior probability is greater than or equal to a preset secondary wake-up threshold, waking up the intelligent voice assistant. According to the intelligent voice assistant awakening method based on bone conduction, whether the voice assistant is triggered or not is determined through double verification, false awakening caused by external interference is avoided, the false awakening rate is reduced, and meanwhile the awakening accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of audio and video processing and multimedia layout control, and in particular to a method for waking up an intelligent voice assistant based on bone conduction. Background Technology

[0002] Currently, most hearing aids on the market do not have voice interaction capabilities, preventing users from using them as an entry point for voice interaction. However, as a device that needs to be worn long-term, hearing aids naturally possess the attributes to become an audio interaction entry point, making them particularly suitable as a carrier for AI intelligent voice assistants and voice interaction. Voice interaction is typically initiated through a voice wake-up function—activating the voice assistant with a specific wake word, thus initiating subsequent interactions.

[0003] Under current technological conditions, the voice wake-up function of both hearing aids and other wearable devices suffers from numerous problems, severely impacting the user experience: First, it is easily affected by background noise, leading to false wake-ups of the voice assistant; second, it may be triggered by non-wearers, such as when a wake-up word is heard in the surrounding environment, causing the device to be mistakenly activated; third, in high-noise environments, the wake-up rate drops significantly, making it difficult to accurately respond to the wearer's commands. These problems directly result in the current situation of low wake-up rate and high false wake-up rate of voice assistants, which not only reduces the user experience but also hinders the widespread application of voice wake-up functions. Summary of the Invention

[0004] The present invention aims to provide a bone conduction-based method for waking up a smart voice assistant, in order to solve the above-mentioned technical problems, avoid false wake-ups caused by external interference, reduce the false wake-up rate, and improve the accuracy of wake-up.

[0005] To address the aforementioned technical problems, this invention provides a method for waking up an intelligent voice assistant based on bone conduction, comprising the following steps:

[0006] Acquire bone conduction signals;

[0007] If the bone conduction signal is determined to be a speech segment, then the original sound signal is acquired;

[0008] Feature extraction is performed on the original sound signal to obtain a sound feature array;

[0009] The posterior probability is calculated based on the sound feature array. If the posterior probability is greater than or equal to a preset wake-up threshold, the posterior probability is calculated based on the bone conduction signal. If the posterior probability is greater than or equal to a preset wake-up threshold, the intelligent voice assistant is woken up.

[0010] In the above scheme, a posterior probability is calculated based on the sound feature array. If the posterior probability is greater than or equal to a preset wake-up threshold, the sound signal is determined to contain a wake word. This ensures that the bone conduction signal corresponding to the sound signal with a high probability of containing a wake word proceeds to the next step, reducing invalid processing. Then, a secondary verification is performed using the bone conduction signal. This double verification determines whether to trigger the voice assistant, effectively avoiding false wake-ups caused by external interference and improving wake-up accuracy while reducing the false wake-up rate.

[0011] Furthermore, it also includes:

[0012] If it is determined that the bone conduction signal is not a speech segment, then the bone conduction signal is reacquired.

[0013] In the above scheme, by continuously monitoring bone conduction signals, subsequent sound signal acquisition is only initiated when a speech segment is detected, which can effectively avoid meaningless subsequent processing.

[0014] Further, if the bone conduction signal is determined to be a speech segment, then the original sound signal is acquired; including:

[0015] The bone conduction signal is segmented into frames to obtain several bone conduction audio frames;

[0016] For any bone conduction audio frame, calculate the short-time energy corresponding to that bone conduction audio frame;

[0017] If the short-time energy corresponding to the bone conduction audio frame is greater than or equal to the preset energy threshold, then the bone conduction audio frame corresponding to the short-time energy is determined to be a speech frame.

[0018] If the number of adjacent and consecutive speech frames is greater than or equal to the preset speech segment threshold, the bone conduction signal is determined to be a speech segment, and the original sound signal is collected.

[0019] In the above scheme, the continuous bone conduction signal is first segmented into several bone conduction audio frames, transforming the originally difficult-to-process long-term signal into audio frames that can be analyzed frame by frame, providing a basic processing unit for subsequent short-time energy calculation and speech segment judgment. Next, by quantifying the short-time energy corresponding to each bone conduction audio frame and combining it with a preset energy threshold, the frame is distinguished as either a speech frame or a non-speech frame, achieving accurate classification of the bone conduction audio frames and providing a basis for subsequent speech segment judgment. Subsequently, based on a preset speech segment threshold, it is determined whether adjacent and continuous speech frames can form a speech segment. If it is determined to be a speech segment, the original sound signal acquisition is initiated, avoiding resource waste caused by invalid acquisition, while also reducing system power consumption and the risk of false triggering.

[0020] Further, the step of extracting features from the original sound signal to obtain a sound feature array includes:

[0021] Acquire the raw sound signal;

[0022] The original audio signal is processed by frame segmentation and windowing to obtain a single frame audio signal;

[0023] Obtain the power spectrum corresponding to a single frame of audio signal;

[0024] Based on the Mel filter bank, Fbank features are extracted from the power spectrum to obtain the sound feature array.

[0025] In the above scheme, the continuous raw audio signal is segmented into frames and windowed, which transforms the long-time signal into a short-time, stationary single-frame audio signal. Windowing also reduces spectral distortion caused by inter-frame truncation, improving the accuracy of subsequent feature extraction and facilitating subsequent frequency domain analysis. Next, the power spectrum corresponding to the single-frame audio signal is obtained, providing a frequency domain basis for subsequent Mel filtering. Finally, Fbank feature extraction is performed on the power spectrum using a Mel filter bank to obtain an audio feature array, providing structured and highly discriminative input features for subsequent feature refinement.

[0026] Further, obtaining the power spectrum corresponding to a single frame of sound signal includes:

[0027] Perform FFT calculation on a single frame of audio signal to obtain the frequency domain data corresponding to the single frame of audio signal;

[0028] Based on the frequency domain data corresponding to a single frame of audio signal, the power spectrum corresponding to the frequency domain data is obtained.

[0029] In the above scheme, the single-frame audio signal is converted from the time domain to the frequency domain through FFT calculation, obtaining the frequency domain data corresponding to the single-frame audio signal, which provides the frequency domain basis for subsequent power spectrum calculation. Then, the power spectrum is obtained through the frequency domain data corresponding to the single-frame audio signal, which can quantify the energy intensity of each frequency component and provide input for the subsequent extraction of Fbank features by the Mel filter bank.

[0030] Furthermore, the step of extracting Fbank features from the power spectrum based on the Mel filter bank to obtain a sound feature array includes:

[0031] Creating Mel filter banks based on power spectrum;

[0032] Based on the power spectrum and Mel filter bank, obtain the Mel filter energy array;

[0033] Logarithmic compression is performed on the energy values ​​in the Mel filter energy array to obtain Fbank eigenvalues;

[0034] By concatenating consecutive Fbank feature values, a sound feature array is obtained.

[0035] In the above scheme, a Mel filter bank is created using the power spectrum, forming a filter bank covering the entire frequency domain, which can accurately filter out key frequency bands related to sound perception. Next, the energy value corresponding to each filter is obtained and logarithmically compressed, enhancing the ability to distinguish low-energy sound components. Subsequently, the obtained Fbank feature values ​​are concatenated to form a sound feature array, which retains both single-frame spectral features and integrates the temporal dynamic information of the sound, providing complete input features for subsequent feature deepening processes and ensuring the capture of wake words.

[0036] Further, the step of calculating a posterior probability based on the sound feature array, and if the posterior probability is greater than or equal to a preset first wake-up threshold, then calculating a second posterior probability based on the bone conduction signal, and if the second posterior probability is greater than or equal to a preset second wake-up threshold, then waking up the intelligent voice assistant; includes:

[0037] Perform feature enhancement on the sound feature array to obtain the enhanced feature array;

[0038] Based on the enhanced feature array, obtain the attention weight array corresponding to the enhanced feature array;

[0039] The attention weight array is used to perform a weighted summation of the enhanced feature array to obtain a weighted feature vector.

[0040] Based on fully connected layers and weighted feature vectors, the probability of one wake-up and the probability of one non-wake-up are obtained;

[0041] Based on the probability of waking up once and the probability of not waking up once, a posterior probability is calculated. If the posterior probability is greater than or equal to a preset threshold for waking up once, a secondary posterior probability is calculated based on the bone conduction signal. If the secondary posterior probability is greater than or equal to a preset threshold for waking up twice, the smart voice assistant is woken up.

[0042] In the above scheme, feature enhancement is performed on the sound feature array to extract a enhanced feature array, which provides more effective input features for subsequent acquisition of attention weight arrays and determination of whether the intelligent voice assistant is activated. Next, attention weights are calculated for each frame of enhanced features in the enhanced feature array. These weights quantify the importance of each frame for wake word recognition, and the enhanced features of each frame are weighted and summed based on the attention weights to obtain a weighted feature vector. This vector contains the core information most relevant to the wake word from all frames and suppresses interference from irrelevant frames, making the weighted feature vector more targeted and discriminative. Then, the weighted feature vector is processed through a fully connected layer to complete the mapping from the weighted feature vector to the wake-up probability, providing the raw values ​​for the subsequent posterior probability calculation. Finally, the posterior probability is calculated and compared with a preset wake-up threshold. If the threshold is met, it indicates that the sound signal likely contains a wake word, triggering a secondary verification based on bone conduction signals. After calculating the secondary posterior probability, it is compared with the preset secondary wake-up threshold. The intelligent voice assistant is only woken up when both the primary and secondary posterior probabilities meet the corresponding wake-up threshold, which ensures both wake-up sensitivity and improves wake-up accuracy.

[0043] Furthermore, the step of deepening the sound feature array to obtain a deepened feature array includes:

[0044] Local temporal features are extracted from the sound feature array based on the first-layer time-delay neural network to obtain the first sound feature array;

[0045] The second feature array of sound is obtained by performing temporal correlation enhancement on the first feature array of sound based on the second-layer time-delay neural network.

[0046] The second feature array of sound is abstracted globally based on the third-layer time-delay neural network to obtain a deeper feature array.

[0047] In the above scheme, the first-layer time-delay neural network maps the sound feature array into a higher-dimensional sound first feature array, which can focus on the local detailed features in the sound feature array, laying the foundation for subsequent deep processing. Then, based on the extraction of the first layer, the second-layer time-delay neural network transforms the sound first feature array into a more abstract sound second feature array, integrating the correlation information between local features. Subsequently, the third-layer time-delay neural network abstracts the sound second feature array into a deepened feature array, accurately capturing the global temporal pattern unique to the wake-up word, providing input for subsequent calculation of wake-up probability.

[0048] Furthermore, the step of obtaining the probability of one wake-up and the probability of one non-wake-up based on the fully connected layer and weighted feature vector includes:

[0049] The weighted feature vector is nonlinearly transformed based on the first fully connected layer to obtain a nonlinear feature vector.

[0050] The nonlinear feature vector is mapped using the second fully connected layer to obtain the probability of one wake-up and the probability of one non-wake-up.

[0051] In the above scheme, the first fully connected layer performs a nonlinear transformation on the weighted feature vector, which can uncover the complex nonlinear correlations in the weighted feature vector and transform it into a more discriminative nonlinear feature vector, providing more effective intermediate features for subsequent probability mapping. Then, the nonlinear feature vector is mapped to a wake-up probability and a non-wake-up probability through the second fully connected layer, realizing a direct mapping from high-order features to classification results. This quantifies the original probability that the current sound signal belongs to the "wake-up" or "non-wake-up" category, providing a basic value for the subsequent calculation of the posterior probability.

[0052] Further, the step of calculating a posterior probability based on a single wake-up probability and a single non-wake-up probability, and calculating a second posterior probability based on bone conduction signals if the first posterior probability is greater than or equal to a preset single wake-up threshold, and waking up the intelligent voice assistant if the second posterior probability is greater than or equal to a preset second wake-up threshold; includes:

[0053] Based on the probability of one wake-up and the probability of one non-wake-up, the posterior probability is calculated. If the posterior probability is greater than or equal to the preset threshold for one wake-up, then the bone conduction signal is subjected to feature extraction to obtain a bone conduction feature array.

[0054] The secondary posterior probability is calculated based on the bone conduction feature array. If the secondary posterior probability is greater than or equal to the preset secondary wake-up threshold, the intelligent voice assistant is woken up.

[0055] In the above scheme, a posterior probability is calculated using a wake-up probability and a non-wake-up probability, quantifying the overall likelihood that a sound signal belongs to the "wake-up" category. When the posterior probability is greater than or equal to a preset wake-up threshold, the sound signal likely contains a wake word, triggering feature extraction of the bone conduction signal to provide input for secondary verification. When the posterior probability is less than the preset wake-up threshold, the sound signal likely does not contain a wake word, and is therefore determined to be non-wake-up, terminating the process. This ensures that only bone conduction signals corresponding to sound signals with a high probability of containing a wake word enter the secondary verification, reducing invalid calculations. Then, feature extraction is performed on the bone conduction signal, and a secondary posterior probability is calculated on the bone conduction feature array using the same steps as for the sound signal, further verifying whether there is a wake-up intent. When the secondary posterior probability is greater than or equal to a preset secondary wake-up threshold, the double verification passes, and the intelligent voice assistant is woken up; if the secondary posterior probability is less than the preset secondary wake-up threshold, the process terminates. Through secondary verification of the bone conduction signal, environmental noise interference is filtered out, fundamentally improving the accuracy of wake-up and reducing the false wake-up rate. Attached Figure Description

[0056] Figure 1 This is a flowchart illustrating a bone conduction-based intelligent voice assistant wake-up method according to an embodiment of the present invention.

[0057] Figure 2 A triangular window diagram of 22 Mel filter banks provided in an embodiment of the present invention;

[0058] Figure 3 An embodiment of the present invention provides a hearing aid in which a bone conduction sensor is placed at the receiver position;

[0059] Figure 4 This invention provides a hearing aid in which a bone conduction sensor is placed on the underside of the hearing aid body, according to one embodiment of the invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] This embodiment incorporates a bone conduction vibration sensor (VPU) (hereinafter referred to as the bone conduction sensor) into the hearing aid. This bone conduction sensor needs to be positioned close to the skin or skull, and can be placed at the bottom of the device or near the speaker (receiver). In addition, the hearing aid in this embodiment also includes a receiver, an air conduction microphone, a main chip, a battery, and external plastic structural components. Because the bone conduction sensor must be in close contact with the skull, it can receive the vibrations generated by the skull when the person speaks, thereby acquiring the corresponding bone conduction signal related to speech. Simultaneously, since the bone conduction sensor can only receive bone conduction signals from its surface and cannot receive sound signals conducted in the air, this characteristic can be used to effectively reduce interference from external environmental noise, allowing the collected bone conduction signals to more accurately assist in determining whether a subsequent intelligent voice assistant has been activated. In terms of hearing aid structure, two layout schemes can both obtain good bone conduction signals: one, see [link to relevant documentation] Figure 3 The bone conduction sensor is placed at the receiver position; the receiver needs to be inserted into the ear canal during use, and placing the bone conduction sensor next to the receiver ensures it fits snugly inside the ear canal, thus effectively collecting the bone conduction signal transmitted through the skull; secondly, see... Figure 4 The bone conduction sensor is placed on the lower side of the hearing aid. When worn, the bone conduction sensor will fit closely to the ear and the temporal bone of the skull, thus effectively picking up bone conduction signals.

[0062] This embodiment is based on the hearing aid itself. The wearer only needs to speak a wake-up word to activate the intelligent voice assistant function in the hearing aid. During the wake-up process, a secondary verification is performed using a bone conduction sensor built into the hearing aid or the receiver in the ear. This secondary verification is compared with the initial wake-up result to determine whether to activate the intelligent voice assistant. This embodiment offers a simple interactive method. Using a hearing aid with a bone conduction sensor can effectively detect whether the wake-up is spoken by the wearer, effectively avoiding external interference, significantly reducing the false wake-up rate, and improving the accuracy of the wake-up.

[0063] This embodiment provides a method for waking up a smart voice assistant based on bone conduction. For detailed steps, please refer to [link to relevant documentation]. Figure 1 ,include:

[0064] Step S1: Acquire bone conduction signals;

[0065] Step S2: If the bone conduction signal is determined to be a speech segment, then the original sound signal is acquired;

[0066] Step S3: Extract features from the original sound signal to obtain a sound feature array;

[0067] Step S4: Calculate the first posterior probability based on the sound feature array. If the first posterior probability is greater than or equal to the preset first wake-up threshold, calculate the second posterior probability based on the bone conduction signal. If the second posterior probability is greater than or equal to the preset second wake-up threshold, wake up the intelligent voice assistant.

[0068] In this embodiment, bone conduction sensors continuously acquire bone conduction signals to monitor whether the user is speaking. Next, it is determined whether the acquired bone conduction signal is a speech segment. If it is, the original sound signal is acquired, and the bone conduction signal is buffered (1 second speech segment) for later use. Then, an air conduction microphone is activated to acquire the original sound signal, and features are extracted from it to obtain a sound feature array, which is then input into the wake-up engine to calculate a posterior probability. If the posterior probability is less than a preset wake-up threshold, meaning the wake-up engine determines that the original sound signal does not contain a wake word, it returns to the bone conduction signal acquisition step. If the posterior probability is greater than or equal to the preset wake-up threshold, meaning the wake-up engine determines that the original sound signal contains a wake word, the bone conduction signal corresponding to a sound signal with a high probability of containing a wake word enters the secondary verification step. Next, the buffered 1 second speech segment is used to perform a secondary wake-up verification on the bone conduction signal to determine whether the bone conduction signal meets the wake-up conditions. If the bone conduction signal verification fails, it returns to the bone conduction signal acquisition step; if the bone conduction signal verification passes, the intelligent voice assistant is activated, completing the wake-up process. The entire process involves initial bone conduction sensing of speaking actions, identifying them as speech segments, and then combining them with the original sound signal to accurately recognize the speech content. A second verification through bone conduction signals forms a dual verification to determine whether to trigger the voice assistant, effectively avoiding false wake-ups caused by external interference. This achieves an accurate and efficient voice wake-up process, reducing the false wake-up rate while improving wake-up accuracy.

[0069] Furthermore, it also includes:

[0070] If it is determined that the bone conduction signal is not a speech segment, then the bone conduction signal is reacquired.

[0071] In this embodiment, by continuously monitoring bone conduction signals, subsequent sound signal acquisition is only initiated when a speech segment is detected, which can effectively avoid meaningless subsequent processing.

[0072] Further, if the bone conduction signal is determined to be a speech segment, then the original sound signal is acquired; including:

[0073] The bone conduction signal is segmented into frames to obtain several bone conduction audio frames;

[0074] For any bone conduction audio frame, calculate the short-time energy corresponding to that bone conduction audio frame;

[0075] If the short-time energy corresponding to the bone conduction audio frame is greater than or equal to the preset energy threshold, then the bone conduction audio frame corresponding to the short-time energy is determined to be a speech frame.

[0076] If the number of adjacent and consecutive speech frames is greater than or equal to the preset speech segment threshold, the bone conduction signal is determined to be a speech segment, and the original sound signal is collected.

[0077] In this embodiment, VAD (Voice Activity Detection) is used to determine whether the detected bone conduction signal is a speech segment: First, the continuous bone conduction signal is divided into several bone conduction audio frames with a fixed frame length, resulting in a series of bone conduction audio frames x. n (where n represents the frame number), transforming long-term signals that are difficult to process directly into audio frames that can be analyzed frame by frame, providing a basic processing unit for subsequent short-time energy calculation and speech segment identification. Next, the short-time energy of each bone conduction audio frame is calculated. Where N is the number of sampling points per frame, x n (k) is the value of the kth sampling point in the nth frame. By summing the squared amplitudes of all sampling points in each frame, the short-time energy corresponding to each bone conduction audio frame is quantified. Combined with a preset energy threshold, the bone conduction audio frame is distinguished as either a speech frame or a non-speech frame, achieving accurate classification of bone conduction audio frames and providing a basis for subsequent speech segment judgment. The preset energy threshold is the average energy over the past 5 seconds. Since each frame contains 512 sampling points at a sampling rate of 16kHz, the duration of each frame is 32ms (512 / 16000 = 32). The past 6.4 seconds correspond to 200 frames (6.4 / 0.032 = 200). Therefore... This involves averaging the short-time energy of the past 200 frames. Then, a preset speech segment threshold of three frames is set, and the short-time energy of a single bone conduction audio frame is judged based on this threshold. If a frame is identified as a speech frame, and there are three or more adjacent and consecutive speech frames, the bone conduction signal is determined to be a speech segment, and the original sound signal acquisition is initiated. If there are fewer than three adjacent and consecutive speech frames, the bone conduction signal is determined to be a non-speech frame, classified as a silence or noise segment. This distinguishes between speech and non-speech segments, avoiding resource waste caused by ineffective acquisition of non-speech segments. The device does not need to run continuously throughout the process, significantly reducing power consumption and minimizing the risk of accidental triggering.

[0078] Further, the step of extracting features from the original sound signal to obtain a sound feature array includes:

[0079] Acquire the raw sound signal;

[0080] The original audio signal is processed by frame segmentation and windowing to obtain a single frame audio signal;

[0081] Obtain the power spectrum corresponding to a single frame of audio signal;

[0082] Based on the Mel filter bank, Fbank features are extracted from the power spectrum to obtain the sound feature array.

[0083] In this embodiment, the continuous original audio signal u is divided into frames, and each frame is windowed (using a Hamming window to smooth edges), i.e., u windowed = u × HammingWindow. This process converts long-time signals into short-time stationary single-frame audio signals and effectively reduces spectral distortion caused by inter-frame truncation through windowing, providing a more reliable foundation for subsequent frequency domain analysis and improving the accuracy of feature extraction. Next, the power spectrum corresponding to the single-frame audio signal is obtained, providing a frequency domain basis for subsequent Mel filtering. Finally, Fbank feature extraction is performed on the power spectrum using a Mel filter bank. The resulting audio feature array has structured and highly discriminative characteristics, providing high-quality input for subsequent feature enhancement.

[0084] Further, obtaining the power spectrum corresponding to a single frame of sound signal includes:

[0085] Perform FFT calculation on a single frame of audio signal to obtain the frequency domain data corresponding to the single frame of audio signal;

[0086] Based on the frequency domain data corresponding to a single frame of audio signal, the power spectrum corresponding to the frequency domain data is obtained.

[0087] In this embodiment, the single-frame audio signal is converted from the time domain to the frequency domain by FFT (Fast Fourier Transform), i.e., U[k] = FFT(u windowed The FFT result is obtained by taking the amplitude and phase information of a single frame of audio signal, where U[k] is a complex number. This provides the frequency domain basis for subsequent power spectrum calculation. Next, the power spectrum is obtained from the frequency domain data of the single frame of audio signal, and the amplitude squared is taken from the FFT result, i.e., P[k] = |U[k]|. 2 This process yields the power spectrum, which quantifies the energy intensity of each frequency component, providing input for subsequent extraction of Fbank features using the Mel filter bank.

[0088] Furthermore, the step of extracting Fbank features from the power spectrum based on the Mel filter bank to obtain a sound feature array includes:

[0089] Creating Mel filter banks based on power spectrum;

[0090] Based on the power spectrum and Mel filter bank, obtain the Mel filter energy array;

[0091] Logarithmic compression is performed on the energy values ​​in the Mel filter energy array to obtain Fbank eigenvalues;

[0092] By concatenating consecutive Fbank feature values, a sound feature array is obtained.

[0093] In this embodiment, the specific process of creating a Mel filter bank using power spectrum is as follows: The frequency range is determined using power spectrum. In this embodiment, the lowest frequency is 300Hz and the highest frequency is 8000Hz (corresponding to the Nyquist frequency at a sampling rate of 16kHz. Since the Nyquist frequency is half the sampling frequency, it represents the highest frequency that can be correctly sampled and recovered. Therefore, at a sampling rate of 16kHz, the highest frequency of a single frame audio signal that can be accurately processed at this sampling rate is 8kHz). Next, 24 frequency points are generated, divided into 23 equal parts on the Mel scale (22 filters require 24 boundary points), and then converted back to linear frequency to obtain 24 points: f0 = 300Hz, f1 ≈ 360Hz, f2 ≈ 430Hz, ..., f... 23 =8000Hz. Then create 22 triangular filters, each filter linearly decaying to 0 with f(m) (m = 1, 2, 3, ..., 22) as the center. The center frequencies of the 22 filter groups are: 300.00, 387.11, 482.63, 587.13, 701.23, 825.64, 961.14, 1108.63, 1269.09, 1443.52, 1633.01, 1838.67, 2061.69, 2303.27, 2564.60, 2846.88, 3151.32, 3479.15, 3831.62, 4209.99, 4615.53, 5049.51, in Hz (Hertz). Each adjacent filter overlaps at its center point. Therefore, by superimposing the coverage areas of all filters, a filter bank covering the entire frequency domain can be formed, accurately selecting key frequency bands related to sound perception. Next, the energy value corresponding to each filter is obtained and logarithmically compressed. The energy value corresponding to the m-th filter is: Power spectrum [k] * filter m [n]. After FFT calculation, the first N / 2 frequency points are taken (due to the conjugate symmetry of FFT, only the positive frequency part needs to be considered), and these are divided into 22 filter sub-bands. The energy of each sub-band is calculated independently. Here, N represents the number of sampling points when performing FFT (Fast Fourier Transform) on each single frame of audio signal. Please refer to... Figure 2Where amplitude represents magnitude, frequency represents frequency, and 22 Mel Filterbank Triangular Windows represents the triangular window diagram of 22 Mel filter banks. Since each filter in the filter bank is triangular, its center frequency is f(m). The gain response coefficient at the center frequency is 1 and decreases linearly towards 0 until it reaches the center frequency of two adjacent filters, at which point the gain response coefficient is 0. Furthermore, the interval between each f(m) widens as the value of m increases. Therefore, the filter m[n] is calculated as follows:

[0094]

[0095] In the formula, k represents the current frequency point. Then, the energy E[m] calculated by each filter is logarithmically compressed to obtain the Fbank feature value: Fbank[m] = log(E[m]), which enhances the ability to distinguish low-energy sound components. Finally, the obtained Fbank feature values ​​are concatenated. In this embodiment, each frame contains 512 sampling points, so the Fbank feature values ​​of 20 consecutive frames are selected and combined into a 20*22 sound feature array. This preserves the single-frame spectral features and integrates the temporal dynamic information of the sound, providing complete input features for the subsequent feature deepening process and ensuring effective capture of wake-up word features.

[0096] Further, the step of calculating a posterior probability based on the sound feature array, and if the posterior probability is greater than or equal to a preset first wake-up threshold, then calculating a second posterior probability based on the bone conduction signal, and if the second posterior probability is greater than or equal to a preset second wake-up threshold, then waking up the intelligent voice assistant; includes:

[0097] Perform feature enhancement on the sound feature array to obtain the enhanced feature array;

[0098] Based on the enhanced feature array, obtain the attention weight array corresponding to the enhanced feature array;

[0099] The attention weight array is used to perform a weighted summation of the enhanced feature array to obtain a weighted feature vector.

[0100] Based on fully connected layers and weighted feature vectors, the probability of one wake-up and the probability of one non-wake-up are obtained;

[0101] Based on the probability of waking up once and the probability of not waking up once, a posterior probability is calculated. If the posterior probability is greater than or equal to a preset threshold for waking up once, a secondary posterior probability is calculated based on the bone conduction signal. If the secondary posterior probability is greater than or equal to a preset threshold for waking up twice, the smart voice assistant is woken up.

[0102] In this embodiment, two wake-up engines employing the same algorithm are used for wake-up discrimination based on sound signals and bone conduction signals, respectively. Both wake-up engines utilize a TDNN architecture. The TDNN architecture is specifically designed for processing temporal speech data and is suitable for wake-up word recognition tasks based on Fbank features. First, a sliding window is used to continuously extract sound features from the streaming audio, which is then used as input to the 20*22 sound feature array (actually 20 consecutive frames of Fbank feature values) obtained in the previous step. This corresponds to a 640ms audio segment (32ms per frame, for a total of 20 frames), forming a frame block stream. Since future frame information is not referenced across windows during processing, the TDNN architecture used possesses strict causal reasoning capabilities. Next, the sound feature array is further refined based on the TDNN architecture, and the refined feature array is extracted, providing more effective input for subsequent acquisition of the attention weight array and determination of whether the intelligent voice assistant is woken up. Then, the refined feature array H = [h] of the [20×128] output from the previous step is processed. 1, h 2, h 3, ...,h 20 ]∈R 20×128 The input linear layer undergoes a linear transformation to obtain the initial attention weight score ε for each frame. t , i.e. ε t =w t *h t , where h t w represents the TDNN output feature of frame t. t Let represent the fully connected linear transformation parameters of frame t. Then, this attention weight score is input into a softmax activation function to obtain the normalized attention weights. Where T represents the total number of frames in the deepened feature array. α t This represents the attention weight for frame t, and the sum of the weights for all frames is 1. The importance of each frame for wake word recognition is quantified by calculating the corresponding attention weight for each frame's enhanced features in the enhanced feature array. Then, the [20x128] enhanced feature array output by the TDNN is weighted and summed based on the attention weights. A weighted feature vector of dimension [1*128] is obtained. This weighting mechanism automatically focuses on frames where the wake word is pronounced, giving higher weights to frames with wake words and reducing the weights of noisy or non-wake word frames. This aggregates the core information most relevant to the wake word from all frames and suppresses interference from irrelevant frames. Compared to pooling, this method is more selective and noise-resistant, making the weighted feature vector more targeted and discriminative. Then, the weighted feature vector is processed through a fully connected layer to complete the mapping from the weighted feature vector to the wake-up probability, providing the raw values ​​for the subsequent calculation of the posterior probability. Finally, the posterior probability is calculated and compared with a preset wake-up threshold. If the threshold is met, it indicates that the sound signal likely contains a wake word, triggering a secondary verification based on bone conduction signals. The secondary posterior probability is calculated and compared with a preset secondary wake-up threshold. The intelligent voice assistant is only activated when both the primary and secondary posterior probabilities meet the corresponding wake-up thresholds. This ensures both wake-up sensitivity and accuracy.

[0103] Furthermore, the step of deepening the sound feature array to obtain a deepened feature array includes:

[0104] Local temporal features are extracted from the sound feature array based on the first-layer time-delay neural network to obtain the first sound feature array;

[0105] The second feature array of sound is obtained by performing temporal correlation enhancement on the first feature array of sound based on the second-layer time-delay neural network.

[0106] The second feature array of sound is abstracted globally based on the third-layer time-delay neural network to obtain a deeper feature array.

[0107] In this embodiment, the input sound feature array is modeled sequentially through three layers of one-dimensional convolutional neural networks (Time Delay Neural Network, TDNN). Each layer uses a standard continuous-frame convolution method, focusing on capturing local temporal relationships within the current window. The first-layer TDNN uses a receptive field of 5 frames (kernel size = 5, stride = 1) to capture local phoneme structures, such as consonant plosives and nasal sounds. Its specific parameters are: number of input channels: 22, number of output channels: 64, convolution kernel size: 5, number of model parameters: 22 × 5 × 64 = 7040 (kernel) + 64 (bias) = ​​710. By mapping the sound feature array to a higher-dimensional first sound feature array through the first-layer TDNN, the focus can be placed on the local detailed features in the sound feature array, laying the foundation for subsequent deep processing. Then, based on the extraction of the first layer, the second-layer TDNN transforms the first sound feature array into a more abstract second sound feature array, integrating the correlation information between local features. The second-layer time-delay neural network also uses ordinary convolution (kernel size = 3) to extract temporal correlation information. It leverages the short-term stationarity and tonal continuity of sound to effectively distinguish between sound and non-sound, enhancing the anti-interference capability of the time-delay neural network. The specific model parameters are: input channels: 64, output channels: 128, kernel size: 5. The total number of model parameters is: 64 × 3 × 128 = 24576 + 128 = 2470. Subsequently, a third-layer time-delay neural network is used to further deepen the feature abstraction of the temporal dimension, covering the entire 20-frame window's global duration to calculate global sound information. The specific model parameters are: input channels: 128, output channels: 128, kernel size: 3. The total number of model parameters is: 128 × 3 × 128 = 49152 + 128 = 49280. Through the third-layer time-delay neural network, the second feature array of sound is abstracted into a deeper feature array, accurately capturing the unique global temporal pattern of the wake-up word, providing input for subsequent calculations of the wake-up probability. Finally, each convolutional layer is followed by Batch Normalization and ReLU activation functions, which can enhance the nonlinear expression and training stability of the time-delay neural network.

[0108] Furthermore, the step of obtaining the probability of one wake-up and the probability of one non-wake-up based on the fully connected layer and weighted feature vector includes:

[0109] The weighted feature vector is nonlinearly transformed based on the first fully connected layer to obtain a nonlinear feature vector.

[0110] The nonlinear feature vector is mapped using the second fully connected layer to obtain the probability of one wake-up and the probability of one non-wake-up.

[0111] In this embodiment, the first fully connected layer outputs a nonlinear feature vector with ReLU activation function, an input dimension of 256, an output dimension of 128, and a parameter count of 256*128+128=33024. This first fully connected layer performs a nonlinear transformation on the weighted feature vector, uncovering complex nonlinear correlations within it and transforming it into a more discriminative nonlinear feature vector, providing more effective intermediate features for subsequent probability mapping. The second fully connected layer outputs a 2D classification result, representing the probabilities of wake-up and non-wake-up. This layer has an input dimension of 128, an output dimension of 2, and a parameter count of 128*2+2=256. By mapping the nonlinear feature vector to a wake-up probability and a non-wake-up probability through the second fully connected layer, a direct mapping from high-order features to classification results is achieved. This quantifies the original probability that the current sound signal belongs to the "wake-up" or "non-wake-up" category, providing a basic numerical value for subsequent calculation of the posterior probability.

[0112] Further, the step of calculating a posterior probability based on a single wake-up probability and a single non-wake-up probability, and calculating a second posterior probability based on bone conduction signals if the first posterior probability is greater than or equal to a preset single wake-up threshold, and waking up the intelligent voice assistant if the second posterior probability is greater than or equal to a preset second wake-up threshold; includes:

[0113] Based on the probability of one wake-up and the probability of one non-wake-up, the posterior probability is calculated. If the posterior probability is greater than or equal to the preset threshold for one wake-up, then the bone conduction signal is subjected to feature extraction to obtain a bone conduction feature array.

[0114] The secondary posterior probability is calculated based on the bone conduction feature array. If the secondary posterior probability is greater than or equal to the preset secondary wake-up threshold, the intelligent voice assistant is woken up.

[0115] In this embodiment, a posterior probability is calculated by acquiring the probability of one wake-up and the probability of one non-wake-up, quantifying the overall likelihood that the sound signal belongs to the "wake-up" category. Specifically, the probability of one wake-up z is: wakeup And the probability of one non-wake-up z non-wakeup Given two decimal numbers between 0 and 1, a posterior probability is calculated through further post-processing to determine whether to initiate a wake-up call. The final posterior probability P(wake_up) is also a decimal between 0 and 1, representing the probability of being woken up by the current sound signal. Through actual testing and debugging, since the wake-up threshold is between 0.45 and 0.85, a preset wake-up threshold of 0.65 was set after balancing the wake-up rate and false wake-up rate. If the posterior probability is greater than or equal to the preset wake-up threshold, it indicates that the sound signal likely contains a wake word and is judged as sound wake-up. In this case, feature extraction of the bone conduction signal is triggered, and a secondary bone conduction wake-up verification is performed. If the posterior probability is less than the preset wake-up threshold, it indicates that the sound signal likely does not contain a wake word, and is judged as non-wake-up. The next step is terminated, and bone conduction signal acquisition is repeated to ensure that only bone conduction signals corresponding to sound signals with a high probability of containing a wake word enter the secondary verification, reducing invalid calculations. Next, feature extraction is performed on the bone conduction signal, and the secondary posterior probability is calculated on the bone conduction feature array using the same steps as for the sound signal to further verify whether there is a wake-up intention. Specifically, the most recent 30 frames (30*16=960ms) are taken from the cached 1s bone conduction signal, and frame-by-frame calculation is performed. 22-dimensional Fbank feature values ​​are extracted from each frame to form a 30*22 bone conduction feature array. The specific process for extracting Fbank feature values ​​has been explained in the above process of extracting the sound feature array. Then, the 30*22 bone conduction feature array is input into a three-layer TDNN time-delay neural network to obtain a bone conduction deepening feature array. The specific structure and algorithm parameters of the three-layer time-delay neural network have been explained in the above process of "deepening the sound feature array to obtain the deepening feature array". Next, the bone conduction feature array is temporally weighted by the attention weight array corresponding to the bone conduction feature array to obtain a weighted [1x128]-dimensional bone conduction weighted feature vector. For details, please refer to the above process of "weighting and summing the feature array based on the attention weight array to obtain the weighted feature vector". Then, this bone conduction weighted feature vector is input into a fully connected layer. Two fully connected layers output the secondary wake-up probability and secondary non-wake-up probability of bone conduction. For details, please refer to the above process of "obtaining the primary wake-up probability and primary non-wake-up probability based on the fully connected layer and weighted feature vector". Finally, the obtained secondary wake-up probability and secondary non-wake-up probability are calculated using the posterior probability to obtain the final secondary posterior probability. For details, please refer to the above process of "calculating the primary posterior probability based on the primary wake-up probability and primary non-wake-up probability". Finally, through actual testing and debugging, since the secondary wake-up threshold is between 0.85 and 0.99, after balancing the wake-up rate and false wake-up rate, the preset secondary wake-up threshold is set to 0.95.If the secondary posterior probability is greater than or equal to the preset secondary wake-up threshold, the bone conduction signal is highly likely a wake-up signal emitted by the hearing aid wearer, and is judged as bone conduction wake-up, thus activating the intelligent voice interaction assistant. If the secondary posterior probability is less than the preset secondary wake-up threshold, the bone conduction signal is highly unlikely to be a wake-up signal emitted by the hearing aid wearer, and is judged as non-wake-up, returning to the bone conduction signal monitoring stage to wait for the next signal input. Through secondary verification of the bone conduction signal, environmental noise interference is filtered out, fundamentally improving the accuracy of wake-up and reducing the false wake-up rate.

[0116] The hearing aid used in this embodiment has the following advantages compared to traditional hearing aids: 1. Low false wake-up rate: It uses both bone conduction and air conduction microphones for dual sound pickup, has a dual wake-up engine, and significantly reduces the false wake-up rate through secondary verification of sound and bone conduction signals. 2. Higher wake-up rate: By setting the threshold of the sound wake-up engine and using secondary verification of bone conduction signals for high-precision screening, the wake-up rate is effectively improved. Compared to traditional pure air conduction wake-up algorithms, whose thresholds are set too high, many genuine wake-up requests are easily filtered out, resulting in a lower wake-up rate. 3. Stronger resistance to external sound interference: Utilizing the sound pickup characteristics of bone conduction, it only picks up the wearer's own voice, completely eliminating interference from external noise and ambient sound, thus improving the user experience. 4. Lower power consumption: Through VAD (Voice Activity Detection), it can effectively detect whether the wearer is speaking, avoiding triggering subsequent modules due to external sounds, and avoiding the power waste caused by the continuous operation of the wake-up engine, achieving high efficiency and energy saving.

[0117] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for waking up an intelligent voice assistant based on bone conduction, characterized in that, Includes the following steps: Acquire bone conduction signals; If the bone conduction signal is determined to be a speech segment, then the original sound signal is acquired; Feature extraction is performed on the original sound signal to obtain a sound feature array; The posterior probability is calculated based on the sound feature array. If the posterior probability is greater than or equal to a preset wake-up threshold, the posterior probability is calculated based on the bone conduction signal. If the posterior probability is greater than or equal to a preset wake-up threshold, the intelligent voice assistant is woken up.

2. The method for waking up an intelligent voice assistant based on bone conduction according to claim 1, characterized in that, Also includes: If it is determined that the bone conduction signal is not a speech segment, then the bone conduction signal is reacquired.

3. The method for waking up an intelligent voice assistant based on bone conduction according to claim 1, characterized in that, If the bone conduction signal is determined to be a speech segment, then the original sound signal is acquired; including: The bone conduction signal is segmented into frames to obtain several bone conduction audio frames; For any bone conduction audio frame, calculate the short-time energy corresponding to that bone conduction audio frame; If the short-time energy corresponding to the bone conduction audio frame is greater than or equal to the preset energy threshold, then the bone conduction audio frame corresponding to the short-time energy is determined to be a speech frame. If the number of adjacent and consecutive speech frames is greater than or equal to the preset speech segment threshold, the bone conduction signal is determined to be a speech segment, and the original sound signal is collected.

4. The method for waking up an intelligent voice assistant based on bone conduction according to claim 1, characterized in that, The step of extracting features from the original sound signal to obtain a sound feature array includes: Acquire the raw sound signal; The original audio signal is processed by frame segmentation and windowing to obtain a single frame audio signal; Obtain the power spectrum corresponding to a single frame of audio signal; Based on the Mel filter bank, Fbank features are extracted from the power spectrum to obtain the sound feature array.

5. The method for waking up an intelligent voice assistant based on bone conduction according to claim 4, characterized in that, The step of obtaining the power spectrum corresponding to a single frame of sound signal includes: Perform FFT calculation on a single frame of audio signal to obtain the frequency domain data corresponding to the single frame of audio signal; Based on the frequency domain data corresponding to a single frame of audio signal, the power spectrum corresponding to the frequency domain data is obtained.

6. The method for waking up an intelligent voice assistant based on bone conduction according to claim 4, characterized in that, The step of extracting Fbank features from the power spectrum based on the Mel filter bank to obtain a sound feature array includes: Creating Mel filter banks based on power spectrum; Based on the power spectrum and Mel filter bank, obtain the Mel filter energy array; Logarithmic compression is performed on the energy values ​​in the Mel filter energy array to obtain Fbank eigenvalues; By concatenating consecutive Fbank feature values, a sound feature array is obtained.

7. The method for waking up an intelligent voice assistant based on bone conduction according to claim 1, characterized in that, The process includes calculating a posterior probability based on the sound feature array; if the posterior probability is greater than or equal to a preset first wake-up threshold, calculating a second posterior probability based on the bone conduction signal; if the second posterior probability is greater than or equal to a preset second wake-up threshold, then waking up the intelligent voice assistant; including: Perform feature enhancement on the sound feature array to obtain the enhanced feature array; Based on the enhanced feature array, obtain the attention weight array corresponding to the enhanced feature array; The attention weight array is used to perform a weighted summation of the enhanced feature array to obtain a weighted feature vector. Based on fully connected layers and weighted feature vectors, the probability of one wake-up and the probability of one non-wake-up are obtained; Based on the probability of waking up once and the probability of not waking up once, a posterior probability is calculated. If the posterior probability is greater than or equal to a preset threshold for waking up once, a secondary posterior probability is calculated based on the bone conduction signal. If the secondary posterior probability is greater than or equal to a preset threshold for waking up twice, the smart voice assistant is woken up.

8. The method for waking up an intelligent voice assistant based on bone conduction according to claim 7, characterized in that, The step of deepening the sound feature array to obtain a deepened feature array includes: Local temporal features are extracted from the sound feature array based on the first-layer time-delay neural network to obtain the first sound feature array; The second feature array of sound is obtained by performing temporal correlation enhancement on the first feature array of sound based on the second-layer time-delay neural network. The second feature array of sound is abstracted globally based on the third-layer time-delay neural network to obtain a deeper feature array.

9. A method for waking up an intelligent voice assistant based on bone conduction according to claim 7, characterized in that, The method of obtaining the probability of one wake-up and the probability of one non-wake-up based on a fully connected layer and a weighted feature vector includes: The weighted feature vector is nonlinearly transformed based on the first fully connected layer to obtain a nonlinear feature vector. The nonlinear feature vector is mapped using the second fully connected layer to obtain the probability of one wake-up and the probability of one non-wake-up.

10. A method for waking up an intelligent voice assistant based on bone conduction according to claim 7, characterized in that, The method involves calculating a posterior probability based on the probability of a single wake-up and the probability of a single non-wake-up. If the posterior probability is greater than or equal to a preset wake-up threshold, a secondary posterior probability is calculated based on the bone conduction signal. If the secondary posterior probability is greater than or equal to a preset secondary wake-up threshold, the intelligent voice assistant is then activated. This includes: Based on the probability of one wake-up and the probability of one non-wake-up, the posterior probability is calculated. If the posterior probability is greater than or equal to the preset threshold for one wake-up, then the bone conduction signal is subjected to feature extraction to obtain a bone conduction feature array. The secondary posterior probability is calculated based on the bone conduction feature array. If the secondary posterior probability is greater than or equal to the preset secondary wake-up threshold, the intelligent voice assistant is woken up.

Images