A method, device and medium for intelligent wake-up of a voice-interactive plush toy

Abstract

This invention discloses a smart wake-up method, device, and medium for voice-interactive plush toys, relating to the field of voice interaction technology. The method includes: constructing a two-dimensional statistical graph based on the original speech signal; generating a candidate sound source mask set using peak detection and Euclidean distance classification; ranking the candidate sound source masks based on density score, harmonicity score, and prosodic feature score to obtain the main sound source mask; refining and weighting the main sound source mask based on amplitude attenuation ratio and time delay to generate a main sound source soft mask; obtaining a clean speech signal; extracting Mel-frequency cepstral coefficient features from the clean speech signal; performing phoneme decoding using a Hidden Markov Model; detecting wake words and semantic trigger words; and thus determining the wake-up state of the plush toy. This invention achieves accurate decoding of wake words and semantic trigger words through lightweight modeling based on a phoneme-level Hidden Markov Model, avoiding the high computational cost dependence of large-scale neural networks.

Description

Technical Field

[0001] This invention relates to the field of voice interaction technology, and in particular to a method, device and medium for intelligent wake-up of a voice-interactive plush toy. Background Technology

[0002] With the rapid development of voice interaction technology, single-channel speech recognition has been widely applied in smart speakers, in-vehicle systems, and voice assistants. However, in noisy or multi-speaker environments, single-channel processing often struggles to distinguish target speech from interfering speech. To improve the robustness of voice interaction, researchers have proposed multi-channel signal processing methods, such as beamforming, generalized cross-correlation phase transform (GCC-PHAT), and power spectrum-based threshold separation. These methods enhance target speech through multi-microphone arrays, reduce the impact of noise, and improve speech recognition performance to some extent.

[0003] However, existing methods generally suffer from two main shortcomings: first, they rely on a single physical acoustic parameter (such as power or cross-correlation delay) for sound source identification, resulting in insufficient robustness in dynamic environments; second, conventional binary masking methods are prone to losing weak speech components, causing damage to speech integrity. Especially in low-power chip-based voice interaction scenarios for plush toys, computational power and energy consumption are limited, making it difficult to directly apply computationally complex methods such as deep neural networks. Therefore, how to achieve sound source separation and wake-up recognition that approximates human auditory mechanisms under limited computational conditions has become a core problem that urgently needs to be solved. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an intelligent wake-up method for voice-interactive plush toys to solve the problem of accurate wake-up recognition during voice interaction with plush toys.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a smart wake-up method for a voice-interactive plush toy, comprising,

[0008] The acquired raw speech signals were subjected to short-time Fourier transform and processed using a bandpass filter bank to obtain the time spectrum and cochlear envelope of the raw speech signals. Based on the amplitude attenuation ratio and time delay of each time frequency point in the time spectrum, a two-dimensional statistical graph was formed. Local peak detection was performed on the two-dimensional statistical graph to obtain the peak time frequency points. Based on the Euclidean distance of all time frequency points to the peak time frequency points, they were classified and assigned to obtain a set of candidate sound source masks.

[0009] The candidate sound source mask set is ranked comprehensively according to density score, harmonicity score and prosodic feature score, and the candidate sound source mask with the highest ranking is selected as the main sound source mask.

[0010] The main sound source mask is refined and weighted based on the amplitude attenuation ratio and time delay to generate a soft mask for the main sound source and obtain a clean speech signal.

[0011] Using a hidden Markov model with preset wake words and semantic trigger words, a pure phoneme sequence of pure speech signal is output. The pure phoneme sequence is compared with the basic phoneme set, and the wake-up state of the plush toy is determined based on the comparison result.

[0012] As a preferred embodiment of the intelligent wake-up method for the voice-interactive plush toy described in this invention, the specific steps for classifying and allocating data based on the Euclidean distance from all time-frequency points to the peak time-frequency point are as follows:

[0013] Calculate the Euclidean distance from all time-frequency points to the peak time-frequency point, and calculate the mean and standard deviation of the Euclidean distance;

[0014] A similarity threshold is set based on the mean and standard deviation of the Euclidean distance;

[0015] If the Euclidean distance from the time-frequency point to the peak time-frequency point is greater than the similarity threshold, the sound source corresponding to the time-frequency point is determined to be an interference sound source; otherwise, the sound source corresponding to the peak time-frequency point is classified into the candidate sound source to which the peak time-frequency point belongs.

[0016] As a preferred embodiment of the intelligent wake-up method for the voice-interactive plush toy described in this invention, the cochlear envelope diagram refers to inputting the original voice signal into a filter bank composed of several equivalent rectangular bandwidth filters and outputting the bandpass signal of each equivalent rectangular bandwidth filter.

[0017] Perform a Hilbert transform on each bandpass signal to obtain the bandpass analytic signal, and use the magnitude of the bandpass analytic signal as the energy envelope value of each bandpass;

[0018] Using the bandpass index as the vertical axis and the time frame index as the horizontal axis, the energy envelope values ​​of all bandpasses are combined to form a cochlear envelope map.

[0019] In a preferred embodiment of the intelligent wake-up method for the voice-interactive plush toy described in this invention, the steps for forming the two-dimensional statistical graph are as follows:

[0020] Based on the time spectrum of the original speech signal, calculate the amplitude attenuation ratio at each time frequency point and the time delay corresponding to the phase difference;

[0021] The amplitude attenuation ratio and time delay at each time frequency point are used as two-dimensional features and mapped onto a two-dimensional plane to form a two-dimensional statistical graph.

[0022] As a preferred embodiment of the intelligent wake-up method for the voice-interactive plush toy described in this invention, the specific steps for comprehensively evaluating the candidate sound source mask set according to density score, harmonicity score, and prosodic feature score are as follows:

[0023] For each candidate sound source mask in the candidate sound source set, the number of time-frequency points is counted and normalized to obtain the density score;

[0024] The inverse short-time Fourier transform is performed on the time-frequency points corresponding to the candidate sound source mask to obtain the reconstructed speech signal corresponding to the candidate sound source mask. The Fourier transform is performed on the autocorrelation sequence of the reconstructed speech signal to obtain the spectral correlation function. The fundamental frequency and harmonic frequencies in the spectral correlation function are weighted and summed to obtain the harmonicity score.

[0025] Based on the prosodic features of human speech and the laws of auditory perception, a target modulation frequency band is set, and the prosodic feature score is obtained by calculating the energy ratio of the cochlear envelope map within the target modulation frequency band to the full modulation frequency band.

[0026] The density score, harmonicity score, and prosodic feature score are weighted and summed to obtain the comprehensive evaluation value of the candidate sound source mask.

[0027] As a preferred embodiment of the intelligent wake-up method for the voice-interactive plush toy described in this invention, the specific steps for outputting a pure phoneme sequence of a pure voice signal are as follows:

[0028] The clean speech signal is divided into frames. The power spectrum of each frame is calculated after being weighted by a Hamming window, and the speech feature vector is output through a Mel filter bank.

[0029] By constructing a phoneme mapping table, the pre-set wake words and semantic trigger words of plush toys are transcribed into a set of basic phonemes;

[0030] For each phoneme in the basic phoneme set, a hidden Markov model is constructed.

[0031] Collect and process a set of training speech samples, including wake words and semantic trigger words, to obtain a set of training speech feature vectors;

[0032] The Hidden Markov Model for each basic phoneme is trained based on the training phoneme set and the training speech feature vector set.

[0033] Based on the speech feature vectors of clean speech signals and the Hidden Markov Model, the Viterbi algorithm is used to search for the optimal state path.

[0034] Based on the correspondence between the state pairs of each Hidden Markov Model and the basic phonemes, the optimal state path is transformed into a pure phoneme sequence.

[0035] As a preferred embodiment of the intelligent wake-up method for the voice-interactive plush toy described in this invention, the method involves comparing a pure phoneme sequence with a basic phoneme set, and determining the wake-up state of the plush toy based on the comparison result. The specific steps are as follows.

[0036] If the pure phoneme sequence contains only the phoneme of the wake word, the plush toy is determined to be in a light wake-up state and a prompt is given. If the new pure phoneme sequence received within a short time window contains the phoneme of the semantic trigger word, the toy will interact with the user via voice according to the function corresponding to the semantic trigger word. Otherwise, it will enter a standby state.

[0037] If the pure phoneme sequence includes phonemes of wake words and semantic trigger words, then the voice interaction with the user will be carried out directly according to the function corresponding to the semantic trigger word;

[0038] If the pure phoneme sequence does not include phonemes of wake words and semantic trigger words, no interaction will be performed, and the system will remain in standby mode.

[0039] As a preferred embodiment of the intelligent wake-up method for the voice-interactive plush toy of the present invention, the original voice signal is synchronously acquired by arranging a dual-microphone array on the left and right sides of the plush toy.

[0040] The original speech signal includes the left original speech signal captured by the left microphone and the right original speech signal captured by the right microphone.

[0041] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the intelligent wake-up method for a voice-interactive plush toy as described in the first aspect of the present invention.

[0042] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the intelligent wake-up method for a voice-interactive plush toy as described in the first aspect of the present invention.

[0043] The beneficial effects of this invention are as follows: This invention constructs time-frequency analysis of dual-channel signals and cochlear envelope maps, and combines amplitude attenuation ratio and time delay consistency to generate and optimize sound source masks, thereby achieving accurate extraction of the main sound source. Through a multi-dimensional scoring mechanism of density, harmonicity and prosodic features, the robustness of the main sound source determination is improved, effectively avoiding the limitations of relying on a single physical parameter. Based on the lightweight modeling of phoneme-level hidden Markov models, accurate decoding of wake words and semantic trigger words is achieved, avoiding the high computational power dependence of large-scale neural networks. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart for the intelligent wake-up method of a voice-interactive plush toy.

[0046] Figure 2 This is a flowchart of the candidate sound source mask generation and classification process.

[0047] Figure 3 Flowchart for evaluating candidate sound source masks and determining the main sound source.

[0048] Figure 4 The flowchart for determining the wake-up state. Detailed Implementation

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0051] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0052] Reference Figures 1-4 As one embodiment of the present invention, this embodiment provides a smart wake-up method for a voice-interactive plush toy, comprising the following steps:

[0053] S1. Perform short-time Fourier transform on the acquired raw speech signal and process it using a bandpass filter bank to obtain the time spectrum and cochlear envelope of the raw speech signal. Based on the amplitude attenuation ratio and time delay of each time frequency point in the time spectrum, form a two-dimensional statistical graph. Perform local peak detection on the two-dimensional statistical graph to obtain the peak time frequency point. Then, classify and assign all time frequency points to the peak time frequency point based on the Euclidean distance from all time frequency points to the peak time frequency point to obtain a set of candidate sound source masks.

[0054] Two microphones are placed on the left and right sides of the plush toy to form a dual-microphone array through spatial spacing. The dual-microphone array synchronously collects the original speech signal, including the left original speech signal collected by the left microphone and the right original speech signal collected by the right microphone. The original speech signal is divided into frames according to a fixed frame length and windowed to obtain the time-domain waveform of each frame. For each frame of the original speech signal, the frequency domain representation is calculated. The time spectrum of the original speech signal is obtained by short-time Fourier transform.

[0055] Among them, the time spectrum refers to the complex value obtained by mapping the original speech signal to the frequency domain after weighting it through a window function. Each frequency point of the time spectrum contains two parts: amplitude and phase. The magnitude of the time spectrum is the amplitude spectrum of the original speech signal, and the angle is the phase spectrum of the original speech signal.

[0056] The original speech signal is input into a filter bank containing several equivalent rectangular bandwidth filters. The complex original speech signal is decomposed into several non-overlapping or partially overlapping frequency bands, and the bandpass signal of each equivalent rectangular bandwidth filter is obtained. By performing Hilbert transform on each bandpass signal, the bandpass analytical signal is obtained. The magnitude of the bandpass analytical signal is used as the energy envelope value of each bandpass, which mainly reflects the intensity fluctuation and rhythm changes of the speech, that is, the prosodic information. The energy envelope values ​​of all bandpasses are combined to form the cochlear envelope map, where the vertical axis is the bandpass index and the horizontal axis is the time frame index.

[0057] It should be noted that a single speech signal contains rich frequency components, and different frequency ranges carry different speech feature information. The bandwidth design of the equivalent rectangular bandwidth filter bank follows the frequency resolution law of the human cochlea (high resolution for low frequencies and low resolution for high frequencies). Therefore, the structure of the speech signal after decomposition by the filter bank is similar to the neural signal processing method before the auditory cortex in the human ear. Therefore, inputting the original speech signal into a filter bank containing several equivalent rectangular bandwidth filters for processing is essentially to simulate the frequency resolution mechanism of the human cochlea, that is, to perform multi-channel decomposition of the original speech signal similar to that of the cochlear basilar membrane.

[0058] Based on the time spectrum of the original speech signal, the amplitude attenuation ratio and the time delay corresponding to the phase difference at each time frequency point are calculated as follows:

[0059] ;

[0060] ;

[0061] In the formula, Indicates the original speech signal in time frames The Amplitude attenuation ratio at each frequency point and These represent the left and right original speech signals in time frames, respectively. The The time spectrum of a frequency point, i.e., the time frequency point. For frequency point index, For time frame indexing, For the original speech signal in time frames The The time delay corresponding to the phase difference at each frequency point represents the time delay caused by the sound wave propagating between the left and right microphones. The left and right original speech signals in time frames The Phase difference at each frequency point and These represent the left and right original speech signals in time frames, respectively. The Phase at each frequency point;

[0062] The amplitude attenuation ratio and time delay at each time frequency point are used as two-dimensional features and mapped onto a two-dimensional plane to form a two-dimensional statistical graph, where the horizontal axis of the two-dimensional statistical graph is the amplitude attenuation ratio and the vertical axis is the time delay.

[0063] Two-dimensional statistical graphs show high-density peaks at the true direction of the sound source, with each peak corresponding to a candidate sound source. Based on the usage scenario of plush toys, it is known that there are several sound sources when a dual-microphone array collects voice signals. The peak search method is used to perform local peak detection on the two-dimensional statistical graph to obtain the time-frequency point corresponding to the peak, i.e. the peak time-frequency point, as well as the amplitude attenuation ratio and time delay.

[0064] Candidate sound source masks are generated by classifying and assigning them based on the Euclidean distances from all time-frequency points to the peak time-frequency point. Specifically, the Euclidean distances from all time-frequency points to the peak time-frequency point are calculated, along with their mean and standard deviation. Using the standard deviation method, a similarity threshold is set based on the mean and standard deviation of the Euclidean distances. By comparing the Euclidean distances with the similarity threshold, if the Euclidean distance from the time-frequency point to the peak time-frequency point is greater than the similarity threshold, the sound source corresponding to that time-frequency point is determined to be an interfering sound source; otherwise, the sound source corresponding to the peak time-frequency point is classified into the candidate sound source category to which the peak time-frequency point belongs, thus generating candidate sound source masks. The calculation formula is as follows:

[0065] ;

[0066] In the formula, As a candidate sound source mask, The Euclidean distance from the time-frequency point to the peak time-frequency point. The similarity threshold;

[0067] By summing up all the candidate sound source masks to which the peak time frequency points belong, a set of candidate sound source masks is obtained;

[0068] It should be noted that the "sound source" in the candidate sound source mask set does not directly refer to the "location or individual from which the sound is emitted" in the physical sense in the real world, but rather to a "set of time and frequency points" aggregated by acoustic features at the signal processing level. The candidate sound source mask is automatically determined by peak detection in a two-dimensional statistical graph, and the entire process relies on the consistency of acoustic features rather than externally set conditions. The peak-driven single-step clustering method avoids the burden on the plush toys in terms of computing power and power consumption. At the same time, Euclidean distance, as a metric, is intuitive, has low computational cost, and strong real-time performance, which meets the low-power real-time processing requirements of toy devices.

[0069] S2. Rank the candidate sound source mask set according to density score, harmonicity score and prosodic feature score, and select the candidate sound source mask with the highest ranking as the main sound source mask.

[0070] For each candidate sound source mask in the candidate sound source set, the number of time-frequency points is counted and normalized to obtain a density score. It should be noted that the larger the number of time-frequency points, the higher the concentration of time-frequency points in the candidate sound source mask, indicating that the speech signal performance of the candidate sound source mask is consistent in different frequency bands and the credibility is stronger.

[0071] Perform inverse short-time Fourier transform on the time-frequency points corresponding to the candidate sound source mask to obtain the reconstructed speech signal corresponding to the candidate sound source mask. Perform Fourier transform on the autocorrelation sequence of the reconstructed speech signal to obtain the spectral correlation function. Perform weighted summation on the fundamental frequency and harmonic frequencies in the spectral correlation function to obtain the harmonicity score. The specific steps are as follows:

[0072] The time spectrum of the original speech signal is subjected to delay summation beamforming to obtain the basic time spectrum of the reconstructed speech signal. Specifically, the microphone with the earlier arrival of the speech signal is selected as the phase reference point. Based on the time delay caused by the propagation of the left and right microphones, the exponential compensation factor is calculated using the frequency domain phase compensation method to perform phase compensation on the speech signal of the other microphone, so that the speech signals of the left and right microphones are aligned. The calculation formula is as follows:

[0073] ;

[0074] In the formula, As an exponential compensation factor, The imaginary unit, Pi This represents the bottom of the index;

[0075] The basic time spectrum is obtained by weighted summation of the time spectrum of the original speech signal based on the exponential compensation factor;

[0076] Based on the fundamental time-frequency spectrum and candidate sound source masks, the candidate time-frequency spectrum corresponding to the candidate sound source masks is calculated. An inverse short-time Fourier transform is performed on the candidate time-frequency spectrum, and the time frames are overlapped and summed to obtain the reconstructed speech signal. The formula for calculating the candidate time-frequency spectrum is as follows:

[0077] ;

[0078] In the formula, For candidate time spectrum, Based on the time spectrum;

[0079] Based on the reconstructed speech signal, the autocorrelation method is used to calculate the autocorrelation sequence of the speech signal, and the calculation formula is as follows:

[0080] ;

[0081] In the formula, This represents the autocorrelation sequence of the speech signal. and For time frames and The amplitude of the reconstructed speech signal For delay, To reconstruct the frame length of the speech signal;

[0082] The spectral correlation function is obtained by performing a Fourier transform on the autocorrelation sequence of the speech signal.

[0083] Harmonic peak detection is performed on the spectral correlation function to obtain the fundamental frequency and harmonic frequencies of the reconstructed speech signal, where the harmonic frequencies are integer multiples of the fundamental frequency;

[0084] The fundamental frequency and the first few harmonic frequencies of the reconstructed speech signal are weighted and summed, and then normalized to obtain the harmonicity score.

[0085] By normalizing the energy envelope value of each bandpass in the cochlear envelope map by applying a moving mean, the normalized envelope of each bandpass is obtained. The normalized envelopes of each bandpass are then linearly aggregated with equal weights to obtain the full bandpass envelope.

[0086] By applying a sliding mean to the full-band envelope to de-trend, the slow drift trend of the full-band envelope is eliminated and the rhythmic fluctuations of the original speech signal are highlighted, resulting in a de-trend envelope. By applying a Hanning window to the de-trend envelope and performing a discrete Fourier transform, the modulation spectrum is obtained.

[0087] Based on the prosodic features of human speech and the laws of auditory perception, a target modulation frequency band range is set, such as 4Hz~16Hz. By performing bandpass filtering integration on the modulation spectrum within the target modulation frequency band range and the full modulation frequency band range respectively, the target band energy and the full band energy are obtained. The prosodic feature score is defined as the ratio of the target band energy to the full band energy.

[0088] The density score, harmonicity score, and prosodic feature score are weighted and summed to obtain the comprehensive evaluation value of the candidate sound source mask. The candidate sound source mask with the highest comprehensive evaluation value is selected as the main sound source mask.

[0089] It should be noted that conventional methods for selecting candidate sound source masks, such as power threshold determination, cross-correlation time delay estimation, and generalized cross-correlation phase transformation, all rely on a single physical acoustic parameter. However, the three-class score fusion mechanism for identifying candidate sound source masks not only considers spatial statistical characteristics but also introduces the physiological and auditory characteristics of speech itself, achieving sound source selection that is closer to the human auditory mechanism. This makes plush toys more resistant to interference in voice interaction.

[0090] S3. Based on the amplitude attenuation ratio and time delay, the main sound source mask is refined and weighted to generate a soft mask for the main sound source and obtain a clean speech signal.

[0091] Based on the amplitude attenuation ratio and time delay of the peak time-frequency point and other time-frequency points in the main sound source mask, the Gaussian weighting method is used to calculate the consistency weight of the time-frequency points. The consistency weight is then used to refine and weight the main sound source mask, where the consistency weight of the peak time-frequency point is 1, resulting in the main sound source soft mask. The formula for calculating the consistency weight is as follows:

[0092] ;

[0093] In the formula, For time and frequency points Consistency weight, and Representing the peak time frequency points The amplitude attenuation ratio and time delay, Indicates the peak frequency point. and The scale parameters representing the amplitude attenuation ratio and time delay are obtained by taking the median absolute deviation of the amplitude attenuation ratio and time delay at all time-frequency points within the main sound source mask.

[0094] The soft mask of the main sound source is multiplied element-wise with the base time spectrum to obtain the soft mask time spectrum. The soft mask time spectrum is then subjected to a short-time Fourier transform and superimposed to obtain the clean speech signal.

[0095] It should be noted that, for voice interaction scenarios involving plush toys, traditional binary masks are prone to accidentally deleting weak speech, causing wake-up failure. However, by using amplitude attenuation ratio and time delay to calculate the consistency weight of time and frequency points, the degree of preservation of each time and frequency point is not a hard choice between two options, but a continuous weight distribution, which is more in line with the auditory characteristics of the human ear and can also take into account both speech integrity and noise suppression.

[0096] S4. Using a hidden Markov model with preset wake words and semantic trigger words, output a pure phoneme sequence of pure speech signal, compare the pure phoneme sequence with the basic phoneme set, and determine the wake-up state of the plush toy based on the comparison result.

[0097] The clean speech signal is segmented into frames. The power spectrum of each frame is calculated after weighting with a Hamming window, and the speech feature vector is output through a Mel filter bank. Specifically, the clean speech signal is segmented into frames, and the power spectrum of each frame is calculated after weighting with a Hamming window. The power spectrum is input into a Mel filter bank, which consists of several Mel bandpass filters. The center frequencies of the filters are uniformly distributed according to the Mel scale. The outputs of each Mel bandpass filter are weighted and summed to obtain the Mel energy. The logarithm of the Mel energy of each frame of clean speech signal is taken, and a discrete cosine transform is performed to obtain the Mel cepstral coefficients of each frame of clean speech signal. The Mel cepstral coefficients of all frames are concatenated into a vector to output the speech feature vector.

[0098] Based on the pinyin rules, the pinyin of the pre-set wake words and semantic trigger words of plush toys (e.g., wake word: maomao, semantic trigger words: play, talk, sing, etc.) are decomposed into initials, finals, and tones. By constructing a pinyin phoneme mapping table, the wake words and semantic trigger words are transcribed into a set of basic phonemes. A training speech set including the wake words and semantic trigger words and the corresponding text information set are obtained. Each text information in the text information set is segmented into words, and each sub-word is converted into pinyin. The training phoneme set is obtained by transing the text information using the pinyin phoneme mapping table. Each training speech in the training speech set is aligned with its corresponding training phoneme to obtain the boundary information of each phoneme in each training speech on the time frame. Each training speech is then labeled to obtain a labeled training speech set. Each labeled training speech is framed, and the power spectrum of each frame signal is calculated after weighting by a Hamming window. The training speech feature vector set is then output through a Mel filter bank.

[0099] For each phoneme in the basic phoneme set, a Hidden Markov Model is constructed. The Hidden Markov Model consists of several states, which represent the initial stage, stable stage and final stage of each phoneme in the pronunciation process. The maximum likelihood estimation method is used to calculate the state transition probability between each state, which describes the possibility of the phoneme transitioning from one stage to the next in the time dimension. Based on the Gaussian distribution, the observation probability distribution of each state is calculated, which describes the probability of the speech feature vector appearing in each state.

[0100] The Hidden Markov Model (HMM) for each basic phoneme is trained based on the training phoneme set and the training speech feature vector set. The model parameters of the HMM are iteratively estimated using the Baum-Welch algorithm (forward-backward algorithm), and the state transition probability and observation probability distribution are iteratively updated. The training ends when the maximum number of iterations is reached.

[0101] Based on the speech feature vector of the clean speech signal and the Hidden Markov Model of the basic phoneme set, the Viterbi algorithm is used to calculate the maximum probability value of reaching each state at each time step by frame-by-frame recursion, and record the optimal transition source to obtain the optimal state path. Specifically, at the beginning, based on the initial state probability, the state transition probability between the first frame of the speech feature vector and each state in the Hidden Markov Model of each basic phoneme is determined. Then, starting from the second frame, the process is recursively repeated. In each frame, the state transition probability from all possible states in the previous frame to the current state is calculated, and combined with the observed probability distribution of the current state, the path with the highest state transition probability is selected as the candidate for the next state path. The predecessor state index of the current state is recorded. Finally, the maximum state transition probability of reaching each state is obtained in the last frame of the speech feature vector, and the globally optimal termination state is selected. From the termination state, the optimal state path is obtained by backtracking step by step along the recorded predecessor state index.

[0102] Furthermore, based on the correspondence between the state of each Hidden Markov Model and the basic phonemes, the optimal state path is converted into a pure phoneme sequence.

[0103] It should be noted that current methods for decoding speech phonemes mainly involve modeling through deep neural networks or modeling based on words and syllables. Both of these require a large amount of speech data for model training. In contrast, this invention uses phonemes as the smallest modeling unit and employs a hidden Markov model to describe the temporal evolution features of phonemes. Decoding can be completed by simply covering the wake word and semantic trigger word, making it more suitable for situations like plush toys where chip power consumption is low, computing power is limited, and data storage space is limited.

[0104] The pure phoneme sequence is compared with the basic phoneme set. Based on the comparison result, the wake-up state of the plush toy is determined. Specifically, if the pure phoneme sequence contains only the phoneme of the wake word, the plush toy is determined to be in a light wake-up state. Within a short time window (e.g., within 5 seconds), it maintains full-power sound reception and provides prompts, such as turning on the light and playing a prompt tone. If the new pure phoneme sequence received within the short time window contains the phoneme of the semantic trigger word, it interacts with the user through voice based on the function corresponding to the semantic trigger word. Otherwise, it enters a standby state. If the pure phoneme sequence includes the phonemes of both the wake word and the semantic trigger word, it directly interacts with the user through voice based on the function corresponding to the semantic trigger word. If the pure phoneme sequence does not include the phonemes of both the wake word and the semantic trigger word, no interaction is performed, and it remains in a standby state.

[0105] This embodiment also provides a computer device applicable to the intelligent wake-up method for voice-interactive plush toys, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the intelligent wake-up method for voice-interactive plush toys as proposed in the above embodiment.

[0106] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0107] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the intelligent wake-up method for voice-interactive plush toys as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0108] In summary, this invention achieves accurate extraction of the main sound source by constructing time-frequency analysis and cochlear envelope maps of dual-channel signals, combining amplitude attenuation ratio and time delay consistency to generate and optimize sound source masks, and improving the robustness of main sound source determination through a multi-dimensional scoring mechanism of density, harmonicity, and prosodic features, effectively avoiding the limitations of relying on a single physical parameter. Based on lightweight modeling of phoneme-level hidden Markov models, it achieves accurate decoding of wake words and semantic trigger words, avoiding the high computational power dependence of large-scale neural networks.

[0109] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for intelligent wake-up of a voice-interactive plush toy, characterized in that: include, The acquired raw speech signals were subjected to short-time Fourier transform and processed using a bandpass filter bank to obtain the time spectrum and cochlear envelope of the raw speech signals. Based on the amplitude attenuation ratio and time delay of each time frequency point in the time spectrum, a two-dimensional statistical graph was formed. Local peak detection was performed on the two-dimensional statistical graph to obtain the peak time frequency points. Based on the Euclidean distance of all time frequency points to the peak time frequency points, they were classified and assigned to obtain a set of candidate sound source masks. The candidate sound source mask set is ranked comprehensively based on density score, harmonicity score, and prosodic feature score. The specific steps are as follows. For each candidate sound source mask in the candidate sound source set, the number of time-frequency points is counted and normalized to obtain the density score; The inverse short-time Fourier transform is performed on the time-frequency points corresponding to the candidate sound source mask to obtain the reconstructed speech signal corresponding to the candidate sound source mask. The Fourier transform is performed on the autocorrelation sequence of the reconstructed speech signal to obtain the spectral correlation function. The fundamental frequency and harmonic frequencies in the spectral correlation function are weighted and summed to obtain the harmonicity score. Based on the prosodic features of human speech and the laws of auditory perception, a target modulation frequency band is set, and the prosodic feature score is obtained by calculating the energy ratio of the cochlear envelope map within the target modulation frequency band to the full modulation frequency band. The density score, harmonicity score, and prosodic feature score are weighted and summed to obtain the comprehensive evaluation value of the candidate sound source mask; The candidate sound source mask with the highest ranking is selected as the main sound source mask; The main sound source mask is refined and weighted based on the amplitude attenuation ratio and time delay to generate a soft mask for the main sound source and obtain a clean speech signal. Using a hidden Markov model with preset wake words and semantic trigger words, a pure phoneme sequence of pure speech signal is output. The pure phoneme sequence is compared with the basic phoneme set, and the wake-up state of the plush toy is determined based on the comparison result.

2. The intelligent wake-up method for a voice-interactive plush toy as described in claim 1, characterized in that: The classification and allocation based on the Euclidean distance from all time-frequency points to the peak time-frequency point is carried out in the following steps. Calculate the Euclidean distance from all time-frequency points to the peak time-frequency point, and calculate the mean and standard deviation of the Euclidean distance; A similarity threshold is set based on the mean and standard deviation of the Euclidean distance; If the Euclidean distance from the time-frequency point to the peak time-frequency point is greater than the similarity threshold, the sound source corresponding to the time-frequency point is determined to be an interference sound source; otherwise, the sound source corresponding to the peak time-frequency point is classified into the candidate sound source to which the peak time-frequency point belongs.

3. The intelligent wake-up method for a voice-interactive plush toy as described in claim 1, characterized in that: The cochlear envelope diagram refers to inputting the original speech signal into a filter bank composed of several equivalent rectangular bandwidth filters, and outputting the bandpass signal of each equivalent rectangular bandwidth filter. Perform a Hilbert transform on each bandpass signal to obtain the bandpass analytic signal, and use the magnitude of the bandpass analytic signal as the energy envelope value of each bandpass; Using the bandpass index as the vertical axis and the time frame index as the horizontal axis, the energy envelope values ​​of all bandpasses are combined to form a cochlear envelope map.

4. The intelligent wake-up method for a voice-interactive plush toy as described in claim 1, characterized in that: The specific steps for forming the two-dimensional statistical chart are as follows: Based on the time spectrum of the original speech signal, calculate the amplitude attenuation ratio at each time frequency point and the time delay corresponding to the phase difference; The amplitude attenuation ratio and time delay at each time frequency point are used as two-dimensional features and mapped onto a two-dimensional plane to form a two-dimensional statistical graph.

5. The intelligent wake-up method for a voice-interactive plush toy as described in claim 1, characterized in that: The specific steps for outputting a clean phoneme sequence of a clean speech signal are as follows. The clean speech signal is divided into frames. The power spectrum of each frame is calculated after being weighted by a Hamming window, and the speech feature vector is output through a Mel filter bank. By constructing a phoneme mapping table, the pre-set wake words and semantic trigger words of plush toys are transcribed into a set of basic phonemes; For each phoneme in the basic phoneme set, a hidden Markov model is constructed. Collect and process a set of training speech samples, including wake words and semantic trigger words, to obtain a set of training speech feature vectors; The Hidden Markov Model for each basic phoneme is trained based on the training phoneme set and the training speech feature vector set. Based on the speech feature vectors of clean speech signals and the Hidden Markov Model, the Viterbi algorithm is used to search for the optimal state path. Based on the correspondence between the state pairs of each Hidden Markov Model and the basic phonemes, the optimal state path is transformed into a pure phoneme sequence.

6. The intelligent wake-up method for a voice-interactive plush toy as described in claim 1, characterized in that: The pure phoneme sequence is compared with the basic phoneme set. Based on the comparison results, the awakening state of the plush toy is determined. The specific steps are as follows. If the pure phoneme sequence contains only the phoneme of the wake word, the plush toy is determined to be in a light wake-up state and a prompt is given. If the new pure phoneme sequence received within a short time window contains the phoneme of the semantic trigger word, the toy will interact with the user via voice according to the function corresponding to the semantic trigger word. Otherwise, it will enter a standby state. If the pure phoneme sequence includes phonemes of wake words and semantic trigger words, then the voice interaction with the user will be carried out directly according to the function corresponding to the semantic trigger word; If the pure phoneme sequence does not include phonemes of wake words and semantic trigger words, no interaction will be performed, and the system will remain in standby mode.

7. The intelligent wake-up method for a voice-interactive plush toy as described in claim 1, characterized in that: The original voice signal was synchronously acquired by arranging a dual-microphone array on the left and right sides of the plush toy. The original speech signal includes the left original speech signal captured by the left microphone and the right original speech signal captured by the right microphone.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the intelligent wake-up method for the voice-interactive plush toy according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the intelligent wake-up method for the voice-interactive plush toy according to any one of claims 1 to 7.

Images