A voice interaction recognition method of an intelligent interactive toy

Abstract

This application relates to the field of speech interaction recognition technology, specifically to a speech interaction recognition method for intelligent interactive toys. The method includes: determining adaptive Mel filter banks and adaptive filter weights for each age group based on the spectral characteristics of children's speech signals; calculating the MFCC feature vector of the children's speech signal based on this, comparing it with the MFCC feature vector of traditional adult speech signals; constructing a total loss function for each age group based on the differences between the feature vectors of adult speech and children's speech at each age group; training the general acoustic model to obtain fine-tuned models for each age group; and performing speech interaction recognition for the toy based on the fine-tuned models for each age group, combined with a teacher-student model. This eliminates the influence of age group differences, improves the accuracy of children's speech recognition, and ensures recognition accuracy while achieving personalized adaptation through the teacher-student model.

Description

Technical Field

[0001] This application relates to the field of voice interaction recognition technology, specifically to a voice interaction recognition method for an intelligent interactive toy. Background Technology

[0002] Intelligent interactive toys are toys that integrate artificial intelligence technology, enabling them to interact with users through functions such as voice recognition, natural language processing, and sentiment analysis, providing personalized learning and entertainment experiences. Voice interaction recognition involves analyzing acoustic speech to obtain corresponding text or pinyin information. This is used to convert user commands into structured, machine-understandable language.

[0003] Traditional voice interaction technologies mostly employ general speech recognition models trained on adult speech data, or further train these models using children's speech data for fine-tuning. However, children's vocal systems differ significantly from adults', and there are also considerable differences between different age groups. Furthermore, different children may have different dialects and catchphrases, among other personalized expressions. Directly training a general speech model could lead to an increased misrecognition rate. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a voice interaction recognition method for intelligent interactive toys, thereby resolving the existing issues.

[0005] The voice interaction recognition method for a smart interactive toy disclosed in this application adopts the following technical solution:

[0006] One embodiment of this application provides a voice interaction recognition method for intelligent interactive toys, the method comprising the following steps:

[0007] Obtain speech datasets of children of different ages;

[0008] Based on the power spectrum characteristics of all frames of speech signals in each children's speech dataset, an adaptive Mel filter bank for each age group is determined; the power spectrum of each frame of speech signal is divided to obtain each bin of the power spectrum of each frame of speech signal; adaptive filter weights of each Mel filter for each bin of each frame of speech signal are constructed according to the power spectrum characteristics of each frame of speech signal; and the MFCC feature vector of each frame of speech signal for each age group is calculated based on the adaptive Mel filter bank and the adaptive filter weights.

[0009] Obtain a general acoustic model for adult speech data and the MFCC feature vectors of each frame of speech signal; based on the difference between the MFCC feature vectors of adult speech data and children's speech datasets of different age groups, construct a total loss function for each age group, train the general acoustic model, and obtain the fine-tuned model for each age group.

[0010] Based on the finely tuned models for each age group, combined with the teacher-student model, voice interaction recognition of toys is performed.

[0011] In one embodiment, the process of obtaining the adaptive Mel filter bank is as follows:

[0012] The initial Mel filter banks for each age group are obtained; the power spectrum of each frame of speech signal for each age group is denoted as the first power spectrum; the first power spectrum is filtered through the current Mel filter banks, and the result is denoted as the second power spectrum; the difference between the first power spectrum and the second power spectrum is calculated as the spectral distance loss of each frame of speech signal; the center frequency and bandwidth of all Mel filters corresponding to each age group are adjusted by an optimization algorithm until the overall spectral distance loss of all frames of speech signal for each age group converges to the minimum, thus obtaining the adaptive Mel filter banks for each age group.

[0013] In one embodiment, the expression for the adaptive filter weights is:

[0014]

[0015] in, The adaptive filter weights of the m-th Mel filter for age group a to the k-th bin in the power spectrum of the t-th frame speech signal are given. The center frequency of the k-th bin in the power spectrum of the speech signal of the t-th frame of age group a; and Age groups The The center frequency and bandwidth of the Mel filter; It is an exponential function with the natural constant as the base.

[0016] In one embodiment, the process of obtaining the MFCC feature vectors of each frame of speech signal for each age group is as follows:

[0017] The power spectrum of each frame of speech signal is filtered by an adaptive Mel filter bank corresponding to the age group. The adaptive filter weights are used as the filter weights when each Mel filter processes the power spectrum of each bin region in the power spectrum of each frame of speech signal. The MFCC feature vector of each frame of speech signal is obtained through the filtering results.

[0018] In one embodiment, the process of obtaining the fine-tuned model is as follows:

[0019] The normalized MFCC feature vectors of all frames of speech signals in the adult speech data are used as input to a general acoustic model. The statistical distribution formed by the features output by the hidden layer in the model is obtained and denoted as the adult feature distribution.

[0020] The standardized MFCC feature vectors of all frames of speech signals in the speech dataset of children of different ages are used as the input of a general acoustic model. The statistical distribution formed by the features output by the hidden layer in the model is obtained and denoted as the feature distribution of children of different ages.

[0021] Calculate the MMD distance between the adult characteristic distribution and the children's characteristic distribution for each age group;

[0022] Construct the total loss function for each age group based on the MMD distance;

[0023] Based on the total loss function, and combined with the standardized MFCC feature vectors of all frames of speech signals for each age group, the general acoustic model is trained to obtain the fine-tuned model for each age group.

[0024] In one embodiment, the expression for the total loss function is:

[0025] ,in, Let a be the total loss function corresponding to age group a; Representing the general acoustic model The underlying loss function used when recognizing children's speech datasets of different age groups; This is the preset balance coefficient; Adult characteristic distribution Distribution of characteristics of children in age group a The MMD distance between them.

[0026] In one embodiment, the process of obtaining the fine-tuned model is as follows:

[0027] The standardized MFCC feature vectors and corresponding labels of all frames of speech signals for each age group are used as input to a general acoustic model for training. The optimization objective is to minimize the total loss function for each age group. The feature extraction layer in the general acoustic model is frozen, and a trainable adaptation layer is added to the general acoustic model architecture. The fine-tuned model for each age group is obtained through training.

[0028] In one embodiment, the finely tuned model based on each age group, combined with a teacher-student model, performs voice interaction recognition for toys, specifically as follows:

[0029] Set up a teacher model in the cloud and a student model on the toy device; for the audio collected by the toy device interaction, calculate the MFCC feature vector of each frame of speech signal in the audio, use the standardized MFCC feature vector as the input of the student model, and output the recognition result.

[0030] The MFCC feature vectors of each frame of speech signal in the audio are synchronously transmitted to the teacher model, and the student model is updated based on the recognition results of the teacher model.

[0031] In one embodiment, updating the student model based on the recognition results of the teacher model includes:

[0032] The standardized MFCC feature vectors of each frame of speech signal are used as input to the teacher model to generate each recognition result and corresponding confidence level for each frame of speech signal. The recognition result with the highest confidence level is taken as the final recognition result for each frame of speech signal. If the confidence level corresponding to the final recognition result exceeds a preset confidence level threshold, the final recognition result is taken as the pseudo-label of the corresponding frame of speech signal. The standardized MFCC feature vectors and pseudo-labels of each frame of speech signal are taken as a high-confidence sample.

[0033] High-confidence samples are stored in a local fine-tuning queue. When the number of high-confidence samples accumulated in the local fine-tuning queue reaches a preset number, or when a preset parameter update time point is reached, the student model's parameters are updated using an optimization algorithm based on the high-confidence samples in the local fine-tuning queue.

[0034] In one embodiment, updating the student model based on the recognition results of the teacher model further includes:

[0035] Every preset number of days, the latest parameters of the teacher model are retrieved from the cloud and the parameters of the student model are updated using a weighted average method. The recognition results of the student model are compared with those of the teacher model. When the recognition accuracy of the student model drops to a preset value or the number of learning samples accumulates to a preset number, the parameters of the student model are reset and online learning restarts.

[0036] This application has at least the following beneficial effects:

[0037] This application targets children's speech data of different age groups. First, it uses an adaptive Mel filter bank to extract acoustic features and standardizes them to eliminate differences between age groups. The filter weights are adaptively determined according to the features of different speech data. Then, based on a general speech recognition model, it measures the feature differences by calculating the MMD distance between adult and children's speech. The model is fine-tuned to learn the feature distribution by minimizing this distance. Finally, a compressed student model is deployed on a toy, and online learning is achieved through synchronization with the teacher model, thus completing personalized adaptation.

[0038] This application employs adaptive feature extraction to preserve key features of children's speech at different ages while eliminating individual differences through standardization. It constructs a regularization term by calculating the MMD distance between adult and children's speech and incorporates this term into the loss function to fine-tune the speech recognition model, enabling it to learn the feature distribution of speech and eliminate the influence of age-related differences. Furthermore, it deploys a compressed student model at the edge to continuously improve recognition accuracy during interaction, while synchronizing with the fully deployed teacher model. This approach balances recognition performance and practical application feasibility while ensuring the computational and storage efficiency of embedded devices. Attached Figure Description

[0039] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a voice interaction recognition method for an intelligent interactive toy provided in this application;

[0041] Figure 2 This is a schematic diagram illustrating the process of acquiring the adaptive Mel filter bank. Detailed Implementation

[0042] To further illustrate the technical means and effects adopted by this application to achieve the intended purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a voice interaction recognition method for an intelligent interactive toy proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0044] The following description, in conjunction with the accompanying drawings, details the specific scheme of the voice interaction recognition method for an intelligent interactive toy provided in this application.

[0045] One embodiment of this application provides a voice interaction recognition method for an intelligent interactive toy.

[0046] Specifically, the following voice interaction recognition method for intelligent interactive toys is provided; please refer to [link / reference]. Figure 1The method includes the following steps:

[0047] Step S1: Obtain speech datasets for children of different age groups.

[0048] To obtain a dataset of children's speech, publicly available children's speech research datasets can be used. In this embodiment, the publicly available dataset used is the ChildMandarin Chinese Dialogue Speech Dataset for children. The children's speech is divided into different sets according to age groups. In this embodiment, age groups are divided according to a physiological development stage of 2 years, thus obtaining children's speech datasets for each age group. Each set contains speech samples of children in the corresponding age group, used for subsequent feature extraction and parameter learning. The purpose of age group division is to ensure that the vocal features of children within the same set are similar.

[0049] Step S2: Determine the adaptive Mel filter bank for each age group based on the power spectrum characteristics of all frames of speech signals in each children's speech dataset; divide the power spectrum of each frame of speech signal to obtain each bin of the power spectrum of each frame of speech signal; construct the adaptive filter weights of each Mel filter for each bin of each frame of speech signal according to the power spectrum characteristics of each frame of speech signal; calculate the MFCC feature vector of each frame of speech signal for each age group based on the adaptive Mel filter bank and the adaptive filter weights.

[0050] Children of different ages exhibit significant physiological differences in their vocal systems. Differences in vocal cord development and resonant cavity size lead to substantial variations in acoustic features such as fundamental frequencies and formant distributions. General speech feature extraction methods, such as fixed-parameter Mel filter banks, cannot accommodate these age-related differences, potentially causing biases in feature extraction and impacting recognition accuracy. Therefore, targeted processing based on the acoustic characteristics of children of different ages is necessary to eliminate feature extraction interference caused by age differences.

[0051] (1) All speech signals in each speech dataset are divided into frames to obtain each frame of speech signal in each speech dataset. The spectrum of each frame of speech signal is obtained by short-time Fourier transform (STFT). Then, the power spectrum of each frame of speech signal is obtained by taking the square of the modulus of each element in the spectrum of each frame of speech signal. STFT is a well-known technology, and the specific process will not be described in detail.

[0052] (2) Based on the power spectrum characteristics of all frames of speech signals in each age group, the center frequency and bandwidth of each Mel filter in the Mel filter bank are adaptively adjusted to obtain the adaptive Mel filter bank for each age group:

[0053] Each filter is determined by its center frequency and bandwidth. The center frequency locates the filter's sensitive center for a specific frequency band, while the bandwidth controls the filter's coverage of signals surrounding that band. By dynamically adjusting these two parameters, the filter bank can be adapted to the acoustic characteristics of children's speech in a specific age group. Because children's speech has relatively richer energy in the high-frequency components (related to the vocal cord development stage), it is necessary to increase the bandwidth of the high-frequency filters to capture more speech details in the high-frequency range, avoid losing key information due to insufficient bandwidth, and ultimately allow the extracted features to more accurately reflect the characteristics of the corresponding age group.

[0054] Specifically, age groups The The center frequency and bandwidth of the Mel filter are denoted as follows: and The center frequency and bandwidth of the filter are adjusted to minimize the speech reconstruction error within the age range. First, the following settings are made: , To obtain the initial values, this application uses the traditional method of constructing a Mel filter bank to obtain the initial Mel filter bank for age group a, thereby obtaining the initial values ​​of the center frequency and bandwidth of each Mel filter for age group a. Then, the power spectrum of each frame of speech signal for age group a is denoted as the first power spectrum. The first power spectrum is filtered through the Mel filter bank under the current parameters, and the result is denoted as the second power spectrum. The values ​​of each frequency in the second power spectrum are used as the predicted values ​​of the corresponding frequencies in the first power spectrum. Then, the minimum mean square error (MSE) is used to calculate the difference between the first power spectrum and the second power spectrum, which is used as the prediction value of each frame of speech signal. Spectral distance loss: This application uses minimizing the overall spectral distance loss of all frames of speech signals in age group a as the optimization objective. The center frequency and bandwidth of all Mel filters are adjusted by gradient descent optimization algorithm. During each optimization, the center frequency and bandwidth of all filters in the Mel filter bank are optimized together until the overall spectral distance loss converges to the minimum value. Finally, the center frequency and bandwidth of each Mel filter are learned, and the adaptive result of the Mel filter bank is obtained, so that the filter bank can match the power spectrum distribution characteristics of age group a, highlight the high-frequency tones unique to children, and weaken age-related pronunciation noise.

[0055] Thus, the center frequency and bandwidth of each adaptive Mel filter in the Mel filter bank for each age group can be obtained through the above method.

[0056] (3) When traditional Mel filter banks filter each frame of speech signal, the fixed filter weights of each Mel filter make it impossible to obtain the speech energy distribution characteristics unique to each age group, which will lead to the loss of key information during feature extraction. Therefore, during filtering, adaptive filter weights should be designed for each Mel filter according to the frequency characteristics of the speech signal to be filtered, so that the main energy frequency bands of speech in each age group can be obtained through the Mel filter bank, which is beneficial for extracting Mel frequency cepstral coefficients (MFCCs) that conform to the characteristics of that age group.

[0057] Specifically, the frequency range of the power spectrum of each frame of speech signal obtained above is divided into equal intervals. In this embodiment, it is divided into 256 equal intervals. That is, equivalent to 256 frequency units; obtain each The center frequency;

[0058] The filter weights for each bin in the power spectrum of each frame of speech signal are calculated using the following expression:

[0059]

[0060] in, The adaptive filter weights of the m-th Mel filter for age group a in the k-th bin of the power spectrum of the t-th frame speech signal are given. The center frequency of the k-th bin in the power spectrum of the speech signal of the t-th frame of age group a; and Age groups The The center frequency and bandwidth of the Mel filter; It is an exponential function with the natural constant as the base.

[0061] The closer the center frequency of the speech signal is to the center frequency of the filter, the greater the weight should be; the larger the filter bandwidth, the slower the weight decay; each Mel filter will target different frequencies. By assigning different weights to the energy, it is possible to extract energy suitable for specific age groups. Acoustic characteristics. Through frequency Discretization enables targeted capture of key frequency components in children's speech, and filter weights are adjusted for each bin to reduce interference from pronunciation differences in feature extraction.

[0062] Furthermore, the three-dimensional matrix composed of the filter weights of all Mel filters in the Mel filter bank for all bins in the power spectrum of all frames of speech signals is denoted as: , The size is ,in, M represents the number of Mel filters in the Mel filter bank, and its value is generally in the range of 20 to 40. In this embodiment, the value of M is set to 20. Let be the number of bins in the power spectrum of all frames of speech signals for age group a. Assuming there are 10 frames of speech signals in age group a, and since the power spectrum of each frame is divided into 256 bins, then... ; Indicates size is The set of real numbers.

[0063] (4) For the speech data of age group a, the speech signal of frame t. Speech signals are obtained by extracting features through an adaptive Mel filter bank. MFCC eigenvectors:

[0064] Take the speech signal of frame t in the speech dataset of age group a. For example, for speech signals The power spectrum is obtained and filtered through an adaptive Mel filter bank for age group a, where the corresponding filter weights obtained in step (3) are used as the parameters for each Mel filter on the speech signal. The filter weights are used when processing the power spectrum of each bin region in the power spectrum. This leads to the acquisition of the speech signal. The power spectrum is filtered, the logarithm of the filtered result is taken, and finally a discrete cosine transform (DCT) is performed. The first 13 dimensions of the coefficients are used to construct the speech signal. The MFCC eigenvectors are calculated using a well-known technique, and the specific process will not be elaborated further.

[0065] Then, the MFCC feature vectors of each frame of speech signal are obtained.

[0066] (5) In order to eliminate the differences in feature distribution among different age groups, the MFCC feature vectors of each frame of speech signal are standardized:

[0067] In the speech dataset for age group a, the mean and standard deviation of the coefficients of the same dimension in the MFCC feature vectors of all frames of speech signals are calculated. The coefficient of that dimension in each frame of speech signal is subtracted from the corresponding mean and then divided by the corresponding standard deviation to obtain the standardized values ​​of the coefficients in each dimension of the MFCC feature vector of each frame of speech signal. This yields the standardized MFCC feature vectors of each frame of speech signal. This maps the feature distributions of different age groups to a unified space, improving the model's robustness to age differences.

[0068] Step S3: Obtain the general acoustic model of adult speech data and the MFCC feature vector of each frame of speech signal; based on the difference between the MFCC feature vectors of adult speech data and children's speech datasets of each age group, construct the total loss function for each age group, train the general acoustic model, and obtain the fine-tuned model for each age group.

[0069] General-purpose speech recognition models are typically trained on adult speech data. If directly applied to children's speech recognition, they suffer from domain shift issues, leading to a significant increase in false recognition rates. This is because the speech characteristics of children at different ages differ considerably from those of adults, causing a significant decrease in the generalization ability of general-purpose speech recognition models on children's speech. Continuing to train the speech recognition model directly using children's data can easily lead to catastrophic forgetting and fails to recognize age-related differences in feature distribution.

[0070] This application, based on the Maximum Mean Difference (MMD) theory, aligns the feature distributions of speech from different age groups by measuring the distance between the feature distributions output from the hidden layer of an adult speech dataset and the feature distributions output from the same hidden layer of a children's speech dataset of the current age group in the Regeneration Kernel Hilbert Space (RKHS). This avoids bias caused by age differences. The specific steps are as follows:

[0071] (1) First, obtain a general acoustic model pre-trained on large-scale adult speech data. ,in, This represents all the basic model parameters included in the general acoustic model. In this embodiment, the Wav2Vec 2.0 speech recognition model is used. Many existing speech recognition models exist, and implementers may use other speech recognition models; this application does not impose specific limitations. Based on this general acoustic model... Based on existing technologies (such as Wav2Vec 2.0), this model already possesses a certain capability for speech feature extraction. Each age group's dataset is used as a training set, and each dataset contains the standardized features processed in step S2.

[0072] (2) Perform feature space alignment: Obtain the standardized MFCC feature vectors of each frame of the adult speech data, where the Mel filter bank is obtained in the traditional way during filtering; use the standardized MFCC feature vectors of all frames of the adult speech data as the input of the general acoustic model, and obtain the statistical distribution formed by the features output by the hidden layer in the model, which is denoted as the adult feature distribution. ;

[0073] Similarly, the normalized MFCC feature vectors of all frames of speech signals in the children's speech dataset of age group a are also used as input to this general acoustic model, and the statistical distribution formed by the features output by the hidden layer in the model is obtained, denoted as the feature distribution of children in age group a. ;

[0074] Calculate the distribution of adult characteristics The MMD distance between the data and the feature distributions of children at each age group is used. By minimizing this MMD distance for each age group, the model learns feature representations that are insensitive to age differences. The distribution is the overall statistical regularity of these hidden layer features. MMD quantifies distributional differences by mapping the data to the Reproducing Kernel Hilbert Space (RKHS) and calculating the difference between the sample means of the two distributions in this space.

[0075] In this application, the distribution of adult characteristics Its function is to serve as a baseline distribution, by comparing it with the characteristic distributions of children at different age groups. A comparison is made to measure the differences between the two. By introducing a regularization term based on this difference into the loss function, the model can be driven to adjust its parameters so that the hidden layer feature distribution of children's speech is closer to that of adults' speech, thereby reducing the model's bias towards age differences and avoiding learning features only for a certain age group.

[0076] (3) Based on the above MMD distance, construct the MMD regularization term to measure the difference in hidden layer feature distribution between adult and child data. Then, combine the MMD regularization term with the standard ASR loss to form the total loss function, which is expressed as:

[0077] ,in, Let a be the total loss function corresponding to age group a; Representing the general acoustic model When recognizing children's speech datasets of various age groups, the basic loss function used in this application embodiment is the cross-entropy loss function; The preset balance coefficient has a value range of 0.1 to 1. In this embodiment, it will be... The value is set to 0.5; To calculate the MMD distance, Adult characteristic distribution Distribution of characteristics of children in age group a The distance between MMDs;

[0078] Used to measure general acoustic models The recognition performance on children's speech datasets of various ages was evaluated, and the general acoustic model was quantified. right The difference between the predicted speech and the true label serves to drive the model to learn the first... The recognition patterns of children's speech across age groups ensure that the model maintains its basic speech recognition capabilities while aligning feature distributions. This, along with the MMD regularization term, forms the optimization objective. It ensures the model accurately recognizes children's speech within the current age group while simultaneously achieving cross-age feature distribution alignment through regularization constraints, thus balancing the model's recognition performance and generalization ability. The labels for each frame of speech signal in the children's speech dataset for each age group are pre-defined.

[0079] (4) Train the general acoustic model. To control computational overhead, freeze the feature extraction layer of the pre-trained model and maintain the basic model parameters in the general acoustic model. Remain unchanged; add two trainable adaptation layers to the model architecture, update the parameters of the last two adaptation layers of the model, and transform the general acoustic model. The parameter output is converted into the fine-tuned model. The parameter output, where, This represents all parameters in the aforementioned adaptation layer.

[0080] The general acoustic model uses a dataset of age group a. During training, from the dataset The normalized MFCC feature vectors and corresponding labels of all frames of speech signals are sampled in batches; the optimization objective is to minimize the total loss function. Forward propagation calculates the ASR loss and MMD loss; during backpropagation, the gradient does not flow through the frozen layer, thus not updating the base model parameters. Only update the adaptation layer parameters The optimizer used is the Adam optimizer, and the learning rate is set to... The training parameters are dynamically adjusted using a cosine annealing strategy; the batch size is set to 16-32 to balance training efficiency and stability. In this embodiment, the batch size is set to 16. In other embodiments of this application, the implementer can set the optimizer, learning rate, batch size, and other parameters according to the actual situation.

[0081] (5) After training, the fine-tuned models for each age group are obtained. The fine-tuned models, including their basic model parameters, are then processed. and adaptation layer parameters Export the data to a format suitable for edge deployment, such as TensorFlow Lite or ONNX. This application exports the data in ONNX format to ensure online adaptive training in subsequent steps.

[0082] Step S4: Based on the finely tuned models for each age group, combined with the teacher-student model, perform voice interaction recognition for toys.

[0083] Although age stratification normalizes feature differences and domain adaptation reduces model age bias, uncertainties in children's speech still exist due to individual real-time differences and dynamic changes in usage scenarios, such as specific catchphrases, temporary pronunciation habits, environmental noise, and interactive context. Pre-trained or offline fine-tuned models struggle to capture these real-time generated personalized features, and long-term use can lead to a decline in recognition accuracy over time due to fixed parameters. Therefore, it is necessary to dynamically adapt to individual differences and environmental changes through online learning during actual interactions to compensate for the shortcomings of the first two steps in terms of real-time performance and personalization.

[0084] This application employs a teacher-student dual-model architecture to further optimize the recognition of children's speech signals received in real-time by toys. The teacher model is a large-scale offline model deployed on a cloud server, responsible for generating high-confidence recognition results for new input speech; the student model is a lightweight edge model deployed on the toy device. When the teacher model's recognition confidence for a certain speech exceeds a threshold, that speech and its corresponding pseudo-label (the teacher model's high-confidence output) are used as training samples. The student model is then fine-tuned online using cross-entropy loss, allowing it to gradually learn the user's personalized speech features. Simultaneously, by periodically synchronizing the teacher model or resetting the student model's parameters, the model avoids forgetting existing knowledge due to continuous learning of new samples, achieving self-supervised real-time optimization while ensuring the computational efficiency of the edge device. The teacher-student model is a well-known existing technique in deep learning, and its specific process will not be elaborated further.

[0085] Specifically, (1) a lightweight student model is deployed on the smart toy device, with its initial parameters derived from the fine-tuned model obtained in step S3; at the same time, a teacher model with a larger number of parameters and stronger performance is deployed on the cloud server as a knowledge source. To meet the requirements of edge computing, the student model is compressed using depthwise separable convolution or quantization techniques. The teacher model retains its complete structure to ensure recognition accuracy and is used to generate high-quality pseudo-labels.

[0086] (2) When a child issues a voice command, the toy device first preprocesses the original audio, such as noise reduction and endpoint detection. Then, it extracts the standardized MFCC feature vectors of each frame of the audio signal. This feature retains the acoustic commonalities of the child's age group and eliminates some individual differences through standardization, providing a good foundation for subsequent personalized learning. The extracted standardized MFCC feature vectors of each frame of the audio signal are then used as input to the student model in the toy device for speech recognition, and the output is the recognition result of each frame of the audio signal. The smart toy reacts according to the recognition result, realizing timely interaction.

[0087] (3) Simultaneously, the standardized MFCC feature vectors of all extracted audio frames are synchronously transmitted to the teacher model in the cloud for recognition, generating each recognition result and corresponding confidence level for each frame of audio signal, and taking the recognition result with the highest confidence level for each frame of audio signal as the final recognition result for each frame of audio signal; the confidence level corresponding to the final recognition result is compared with a preset confidence threshold, wherein, in this embodiment, the confidence threshold is set to 0.9. In other embodiments of this application, the implementer may set the confidence threshold according to the actual situation. If the confidence level corresponding to the final recognition result exceeds the confidence threshold, it indicates that the recognition result is reliable, and the final recognition result of the frame of audio signal is taken as the pseudo-label of the frame of audio signal, and the standardized MFCC feature vector and pseudo-label of the frame of audio signal are taken as a high-confidence sample; if it does not exceed the threshold, the frame of audio signal sample is discarded to avoid introducing incorrect labeling.

[0088] High-confidence samples are stored in a local fine-tuning queue. When the number of high-confidence samples accumulated in the local fine-tuning queue reaches a preset number, or when a preset parameter update time point is reached, the student model parameters are updated using mini-batch stochastic gradient descent (SGD) algorithms based on the high-confidence samples in the local fine-tuning queue. In this embodiment, the preset number of samples is set to 100; the preset parameter update time point refers to setting a parameter update time point every 10 minutes, i.e., updating the student model parameters every 10 minutes. The learning rate is set to a small value; in this embodiment, the learning rate is set to 00.001 to ensure learning stability, and a momentum optimizer, such as Adam, is used to accelerate convergence.

[0089] (4) To prevent the student model from forgetting general knowledge during continuous learning, a periodic synchronization mechanism is set up: soft synchronization: every preset number of days, the latest parameters of the teacher model are pulled from the cloud and the parameters of the student model are updated in a weighted average manner; hard reset: when the toy device detects that the recognition accuracy of the student model has dropped significantly or the learning samples have accumulated to a certain scale, the parameters of the student model are reset to the result of step S3, and online learning is restarted. In this embodiment, the preset number of days is 1 day; the standard for the significant drop in recognition accuracy is: using the recognition result of the teacher model as a benchmark, the audio recognition result of the student model is compared with the audio recognition result of the teacher model every 10 minutes. If the error rate exceeds 15%, it indicates that the recognition accuracy has dropped significantly; the certain scale refers to the accumulation of 100 samples.

[0090] A schematic diagram of the process of acquiring the adaptive Mel filter bank is shown below. Figure 2 As shown.

[0091] In summary, this application's embodiments target children's speech data of different age groups. First, it uses an adaptive Mel filter bank to extract acoustic features and standardizes them to eliminate differences between age groups. The filter weights are adaptively determined based on the features of different speech data. Then, based on a general speech recognition model, it measures feature differences by calculating the MMD distance between adult and children's speech, and fine-tunes the model to learn the feature distribution by minimizing this distance. Finally, it deploys a compressed student model on a toy, enabling online learning through synchronization with the teacher model, thus achieving personalized adaptation.

[0092] This application employs adaptive feature extraction to preserve key features of children's speech at different ages while eliminating individual differences through standardization. It constructs a regularization term by calculating the MMD distance between adult and children's speech and incorporates this term into the loss function to fine-tune the speech recognition model, enabling it to learn the feature distribution of speech and eliminate the influence of age-related differences. Furthermore, it deploys a compressed student model at the edge to continuously improve recognition accuracy during interaction, while synchronizing with the fully deployed teacher model. This approach balances recognition performance and practical application feasibility while ensuring the computational and storage efficiency of embedded devices.

[0093] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0094] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0095] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them; modifications to the technical solutions described in the foregoing embodiments, or equivalent substitutions of some of the technical features, do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A voice interaction recognition method for an intelligent interactive toy, characterized in that, The method includes the following steps: Obtain speech datasets of children of different ages; Based on the power spectrum characteristics of all frames of speech signals in each children's speech dataset, an adaptive Mel filter bank for each age group is determined; the power spectrum of each frame of speech signal is divided to obtain each bin of the power spectrum of each frame of speech signal; adaptive filter weights of each Mel filter for each bin of each frame of speech signal are constructed according to the power spectrum characteristics of each frame of speech signal; and the MFCC feature vector of each frame of speech signal for each age group is calculated based on the adaptive Mel filter bank and the adaptive filter weights. Obtain a general acoustic model for adult speech data and the MFCC feature vectors of each frame of speech signal; based on the difference between the MFCC feature vectors of adult speech data and children's speech datasets of different age groups, construct a total loss function for each age group, train the general acoustic model, and obtain the fine-tuned model for each age group. Based on the finely tuned models for each age group, combined with the teacher-student model, voice interaction recognition of toys is performed. The process of obtaining the fine-tuned model is as follows: The normalized MFCC feature vectors of all frames of speech signals in the adult speech data are used as input to a general acoustic model. The statistical distribution formed by the features output by the hidden layer in the model is obtained and denoted as the adult feature distribution. The standardized MFCC feature vectors of all frames of speech signals in the speech dataset of children of different ages are used as the input of a general acoustic model. The statistical distribution formed by the features output by the hidden layer in the model is obtained and denoted as the feature distribution of children of different ages. Calculate the MMD distance between the adult characteristic distribution and the children's characteristic distribution for each age group; Construct the total loss function for each age group based on the MMD distance; Based on the total loss function, and combined with the standardized MFCC feature vectors of all frames of speech signals for each age group, the general acoustic model is trained to obtain the fine-tuned model for each age group.

2. The voice interaction recognition method for an intelligent interactive toy as described in claim 1, characterized in that, The process of obtaining the adaptive Mel filter bank is as follows: The initial Mel filter banks for each age group are obtained; the power spectrum of each frame of speech signal for each age group is denoted as the first power spectrum; the first power spectrum is filtered through the current Mel filter banks, and the result is denoted as the second power spectrum; the difference between the first power spectrum and the second power spectrum is calculated as the spectral distance loss of each frame of speech signal; the center frequency and bandwidth of all Mel filters corresponding to each age group are adjusted by an optimization algorithm until the overall spectral distance loss of all frames of speech signal for each age group converges to the minimum, thus obtaining the adaptive Mel filter banks for each age group.

3. The voice interaction recognition method for an intelligent interactive toy as described in claim 1, characterized in that, The expression for the adaptive filter weights is: in, The adaptive filter weights of the m-th Mel filter for age group a to the k-th bin in the power spectrum of the t-th frame speech signal are given. The center frequency of the k-th bin in the power spectrum of the speech signal of the t-th frame of age group a; and Age groups The The center frequency and bandwidth of the Mel filter; It is an exponential function with the natural constant as the base.

4. The voice interaction recognition method for an intelligent interactive toy as described in claim 1, characterized in that, The process of obtaining the MFCC feature vectors of each frame of speech signal for each age group is as follows: The power spectrum of each frame of speech signal is filtered by an adaptive Mel filter bank corresponding to the age group. The adaptive filter weights are used as the filter weights when each Mel filter processes the power spectrum of each bin region in the power spectrum of each frame of speech signal. The MFCC feature vector of each frame of speech signal is obtained through the filtering results.

5. The voice interaction recognition method for an intelligent interactive toy as described in claim 1, characterized in that, The expression for the total loss function is: ,in, Let a be the total loss function corresponding to age group a; Representing the general acoustic model The underlying loss function used when recognizing children's speech datasets of different age groups; This is the preset balance coefficient; Adult characteristic distribution Distribution of characteristics of children in age group a The MMD distance between them.

6. The voice interaction recognition method for an intelligent interactive toy as described in claim 1, characterized in that, The process of obtaining the fine-tuned model is as follows: The standardized MFCC feature vectors and corresponding labels of all frames of speech signals for each age group are used as input to a general acoustic model for training. The optimization objective is to minimize the total loss function for each age group. The feature extraction layer in the general acoustic model is frozen, and a trainable adaptation layer is added to the general acoustic model architecture. The fine-tuned model for each age group is obtained through training.

7. The voice interaction recognition method for an intelligent interactive toy as described in claim 1, characterized in that, The finely tuned model based on each age group, combined with a teacher-student model, performs voice interaction recognition for toys, specifically as follows: Set up a teacher model in the cloud and a student model on the toy device; for the audio collected by the toy device interaction, calculate the MFCC feature vector of each frame of speech signal in the audio, use the standardized MFCC feature vector as the input of the student model, and output the recognition result. The MFCC feature vectors of each frame of speech signal in the audio are synchronously transmitted to the teacher model, and the student model is updated based on the recognition results of the teacher model.

8. The voice interaction recognition method for an intelligent interactive toy as described in claim 7, characterized in that, The step of updating the student model based on the recognition results of the teacher model includes: The standardized MFCC feature vectors of each frame of speech signal are used as input to the teacher model to generate each recognition result and corresponding confidence level for each frame of speech signal. The recognition result with the highest confidence level is taken as the final recognition result for each frame of speech signal. If the confidence level corresponding to the final recognition result exceeds a preset confidence level threshold, the final recognition result is taken as the pseudo-label of the corresponding frame of speech signal. The standardized MFCC feature vectors and pseudo-labels of each frame of speech signal are taken as a high-confidence sample. High-confidence samples are stored in a local fine-tuning queue. When the number of high-confidence samples accumulated in the local fine-tuning queue reaches a preset number, or when a preset parameter update time point is reached, the student model's parameters are updated using an optimization algorithm based on the high-confidence samples in the local fine-tuning queue.

9. The voice interaction recognition method for an intelligent interactive toy as described in claim 8, characterized in that, The step of updating the student model based on the recognition results of the teacher model also includes: Every preset number of days, the latest parameters of the teacher model are retrieved from the cloud and the parameters of the student model are updated using a weighted average method. The recognition results of the student model are compared with those of the teacher model. When the recognition accuracy of the student model drops to a preset value or the number of learning samples accumulates to a preset number, the parameters of the student model are reset and online learning restarts.

Images