Children voice recognition method and related equipment

Abstract

The invention is suitable for the technical field of artificial intelligence, and provides a child voice recognition method, which comprises the following steps: acquiring to-be-recognized voice data of a target child user, and acquiring an individual voice portrait of the target child user; based on the individual voice portrait, performing fine tuning on a preset acoustic recognition model to obtain a target acoustic recognition model; and performing voice recognition processing on the to-be-recognized voice data according to the target acoustic recognition model to obtain a voice recognition result of the to-be-recognized voice data. According to the invention, fine tuning is carried out on the preset acoustic recognition model through the individual voice portrait of the target child user to obtain the target acoustic recognition model, and voice recognition processing is carried out on the to-be-recognized voice data of the target child user according to the target acoustic recognition model to obtain the voice recognition result of the to-be-recognized voice data. The problem that an existing voice recognition method is difficult to adapt to the pronunciation mode of a child in a child voice scene, and consequently the recognition error rate is high is solved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a method and related equipment for children's speech recognition. Background Technology

[0002] In children's voice interaction scenarios, plush toys are often used to accompany children in natural speech conversations. However, most current speech recognition (ASR) systems are trained on adult speech data, yet children's speech differs significantly from adult speech in several aspects. Because children's vocal tracts are not fully developed, their pitch, timbre, and articulation patterns exhibit greater instability. For example, children speak quickly, have varied intonation, pronounce inaccurately, and have a smaller and less standardized vocabulary, making it difficult for them to perform well in children's speech scenarios, especially in conversations. The inability to adapt to children's pronunciation patterns leads to a higher recognition error rate.

[0003] Therefore, there is an urgent need for a speech recognition method that can accurately identify children's speech, in order to solve the problem that existing speech recognition methods are difficult to adapt to children's pronunciation patterns in children's speech scenarios, resulting in a high recognition error rate. Summary of the Invention

[0004] This application provides a method for children's speech recognition, which addresses the problem that existing speech recognition methods struggle to adapt to children's pronunciation patterns in specific speech scenarios, leading to high error rates. By using an individual speech profile of the target child user, a pre-set acoustic recognition model is fine-tuned to obtain a target acoustic recognition model. Based on this model, the speech data of the target child user is processed to obtain the speech recognition result. This method solves the problem of existing speech recognition methods failing to adapt to children's pronunciation patterns in specific speech scenarios, resulting in high error rates.

[0005] In a first aspect, embodiments of this application provide a method for recognizing children's speech, the method comprising the following steps:

[0006] Acquire the voice data to be identified from the target child user, and acquire the individual voice profile of the target child user;

[0007] Based on the individual voice profile, the preset acoustic recognition model is fine-tuned to obtain the target acoustic recognition model;

[0008] Based on the target acoustic recognition model, the speech data to be recognized is processed for speech recognition to obtain the speech recognition result of the speech data to be recognized.

[0009] Optionally, obtaining the voice data to be recognized from the target child user includes:

[0010] Obtain the initial voice data of the target child user;

[0011] The initial speech data is subjected to power normalization and spectral subtraction to obtain preprocessed speech data;

[0012] Adaptive frame adjustment is performed on the preprocessed speech data to obtain the speech data to be recognized.

[0013] Optionally, obtaining the individual voice profile of the target child user includes:

[0014] Obtain multiple voice samples from the target child users;

[0015] The voice samples are subjected to feature extraction to obtain the voice features of the target child user, with each voice sample corresponding to one voice feature;

[0016] Based on the speech features and the number of speech samples, an individual speech profile of the target child user is generated.

[0017] Optionally, the step of fine-tuning the preset acoustic recognition model based on the individual speech profile to obtain the target acoustic recognition model includes:

[0018] Based on the individual voice profile, determine the scaling factor and offset factor;

[0019] Based on the scaling factor and the offset factor, the intermediate layer parameters of the acoustic recognition model are fine-tuned to obtain the target acoustic recognition model.

[0020] Optionally, the acoustic recognition model includes a Transformer layer, and the method further includes:

[0021] The original weights of the Transformer layer are frozen, and a low-rank trainable matrix is ​​inserted into the Transformer layer.

[0022] When a model adjustment signal is detected, the low-rank trainable matrix is ​​trained locally using incremental data to obtain the adjusted low-rank trainable matrix.

[0023] The Transformer layer is updated based on the adjusted low-rank trainable matrix.

[0024] Optionally, after updating the Transformer layer based on the adjusted low-rank trainable matrix, the method further includes:

[0025] The gradient parameters obtained from local training are uploaded to the server so that the server can update the global model through federated learning.

[0026] The system receives the global model from the server, deploys the global model locally, and then obtains the updated acoustic recognition model.

[0027] Optionally, the step of performing speech recognition processing on the speech data to be recognized according to the target acoustic recognition model to obtain the speech recognition result of the speech data to be recognized includes:

[0028] Based on the target acoustic recognition model, the speech data to be recognized is processed for speech recognition to obtain multiple candidate speech recognition results;

[0029] Based on the contextual semantics of the candidate speech recognition results, the candidate speech recognition results are scored to obtain a score for each candidate speech recognition result.

[0030] The scores of the candidate speech recognition results are subjected to confidence weighting to obtain multiple confidence weighted scores for the candidate speech recognition results, with each candidate speech recognition result corresponding to one confidence weighted score;

[0031] Among the multiple candidate speech recognition results after confidence weighting, the candidate speech recognition result with the highest confidence weighting score is selected as the speech recognition result of the speech data to be recognized.

[0032] Secondly, embodiments of this application provide a child voice recognition device, the child voice recognition device comprising:

[0033] The acquisition module is used to acquire the voice data to be identified from the target child user, and to acquire the individual voice profile of the target child user;

[0034] The first processing module is used to fine-tune the preset acoustic recognition model based on the individual voice profile to obtain the target acoustic recognition model.

[0035] The second processing module is used to perform speech recognition processing on the speech data to be recognized according to the target acoustic recognition model, so as to obtain the speech recognition result of the speech data to be recognized.

[0036] Thirdly, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the child speech recognition method provided in embodiments of the present invention.

[0037] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the child speech recognition method provided in the embodiments of the present invention.

[0038] The above-mentioned solution of this application has the following beneficial effects: acquiring the speech data to be recognized from the target child user, and acquiring the individual speech profile of the target child user; fine-tuning the preset acoustic recognition model based on the individual speech profile to obtain the target acoustic recognition model; and performing speech recognition processing on the speech data to be recognized according to the target acoustic recognition model to obtain the speech recognition result of the speech data to be recognized. This invention solves the problem that existing speech recognition methods are difficult to adapt to children's pronunciation patterns in children's speech scenarios, resulting in a high recognition error rate.

[0039] Other beneficial effects of this application will be described in detail in the following detailed description section. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating a child speech recognition method provided in one embodiment of this application;

[0042] Figure 2 A schematic diagram of the structure of a child voice recognition device provided in an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0044] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0045] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0046] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0047] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0048] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0049] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0050] like Figure 1 As shown, Figure 1 This is a flowchart of a child speech recognition method provided by an embodiment of the present invention. The child speech recognition method includes the following steps:

[0051] 101. Obtain the voice data to be recognized from the target child user, and obtain the individual voice profile of the target child user.

[0052] In this embodiment of the invention, the above-described children's speech recognition method can be applied to a server, with communication between the server and a smart device. The smart device can be a smart toy, a learning machine, a voice assistant, etc., and includes a voice acquisition device for collecting user voice data during natural interaction. The smart toy is equipped with teaching voices (such as reading aloud and reciting), interactive game voices (such as question-and-answer and dialogue), and daily command voices (such as playing, turning on lights, and checking the weather).

[0053] The target child users mentioned above can be either children who wake up the smart device or children who interact with the smart device.

[0054] The aforementioned voice data to be recognized can be voice data collected in real time by the voice acquisition device of the smart device when the target child user interacts with the smart device. For example, it can be voice data with unclear syllables, unstable speech rate, irregular pauses, and incomplete semantic expression.

[0055] The aforementioned individual voice profile can be constructed by analyzing an individual's voice characteristics to create a digital descriptive model of that individual, which is used to identify and distinguish the voices of different individuals.

[0056] The individual voice profiles of the target child users mentioned above can be digital descriptive models of the target child users constructed based on their voice characteristics.

[0057] It should be noted that the construction of an individual's voice profile can be based on a multi-layered analysis of voice features, including physiological, behavioral, and psychological characteristics. Physiological features can include voiceprints, intonation, and pitch, reflecting characteristics such as age and gender. Behavioral features can include speaking rhythm, word choice patterns, and interaction habits, revealing interests or consumption tendencies. Psychological features can include emotional state and personality traits, which can be inferred from the emotional intensity or stability of the voice.

[0058] 102. Based on individual voice profiles, fine-tune the preset acoustic recognition model to obtain the target acoustic recognition model.

[0059] In this embodiment of the invention, the aforementioned preset acoustic recognition model can be an acoustic model built based on deep learning or machine learning, such as GMM-HMM, GMM-UBM, etc. The aforementioned GMM-HMM models acoustic features using a Gaussian Mixture Model (GMM) and handles temporal dependencies using a Hidden Markov Model (HMM). The aforementioned GMM (Gaussian Mixture Model) is a probability-based statistical model used to describe data generated by a mixture of multiple Gaussian distributions (normal distributions). The core idea of ​​GMM is that complex data distributions can be viewed as a combination of multiple "small groups" or clusters, where data points within each cluster follow a Gaussian distribution, and the entire dataset is a weighted sum of these distributions. The aforementioned HMM (Hidden Markov Model) describes a Markov process containing hidden unknown parameters. The core idea of ​​HMM is that the true state (hidden state) of the system cannot be directly observed, but each state generates an observable output with a certain probability. By analyzing these observable sequences, the characteristics of the hidden state can be inferred or predictions can be made. The aforementioned GMM-UBM (Gaussian Mixture Model-Universal Background Model) is used to extract personalized features from speech and for authentication. The core idea of ​​GMM-UBM is to utilize a universal background model to efficiently model speaker specificity.

[0060] The aforementioned pre-defined acoustic recognition model can extract acoustic features from speech data by simulating human auditory characteristics through MFCC (Mel-Frequency Cepstrum) or Fbank (Filter Bank Energy). The Mel-Frequency Cepstrum is a linear transformation of the logarithmic energy spectrum based on a nonlinear Mel scale of sound frequency. The Mel-Frequency Cepstrum is based on the nonlinear perception of frequency by the human ear (Mel scale), capturing the formant information of speech by transforming the spectrum to the cepstrum domain, thus achieving feature dimensionality reduction. The Filter Bank Energy directly uses the Mel filter bank to filter the power spectrum, calculating the energy of each filter, which more closely approximates the original spectral distribution and retains more details.

[0061] The aforementioned pre-defined acoustic recognition model can map input speech data (such as speech waveforms) to corresponding speech unit sequences (such as phonemes or syllables). The core of the acoustic recognition model is to establish the mapping relationship between acoustic features and speech units. The acoustic recognition model captures the spectral information of speech by extracting the acoustic features of the speech signal, and uses statistical models or deep neural networks to calculate the probability of observed features under a given speech unit.

[0062] The aforementioned fine-tuning can be a process of optimizing the parameters of a preset acoustic recognition model by generating a set of parameters with minimal update cost from individual speech profiles.

[0063] The aforementioned target acoustic recognition model can be obtained by fine-tuning a preset acoustic recognition model using individual voice profiles of target child users. This target acoustic recognition model is capable of recognizing the voice data of target child users.

[0064] 103. Based on the target acoustic recognition model, perform speech recognition processing on the speech data to be recognized to obtain the speech recognition result of the speech data to be recognized.

[0065] In this embodiment of the invention, the aforementioned speech data to be identified may be speech data produced by the target child user with unclear syllables, unstable speech rate, irregular pauses, and incomplete semantic expression.

[0066] The aforementioned speech recognition processing can be a process of converting speech data to be recognized into text or related instructions based on a target acoustic recognition model.

[0067] The aforementioned speech recognition results can be text or related instructions obtained after processing the speech data to be recognized using a target acoustic recognition model.

[0068] It should be noted that, based on the target acoustic recognition model, speech recognition processing is performed on the speech data to be recognized to obtain the speech recognition result of the speech data to be recognized. The target acoustic recognition model can improve the efficiency and accuracy of voice interaction.

[0069] In this embodiment of the invention, the invention uses the individual voice profile of the target child user to fine-tune a preset acoustic recognition model to obtain a target acoustic recognition model. Based on the target acoustic recognition model, the speech data of the target child user to be recognized is processed for speech recognition, thereby obtaining the speech recognition result of the speech data to be recognized. This can improve the efficiency and accuracy of voice interaction and solve the problem that existing speech recognition methods are difficult to adapt to children's pronunciation patterns in children's voice scenarios, resulting in a high recognition error rate.

[0070] In this embodiment of the invention, the speech data to be recognized from the target child user is acquired, as well as an individual speech profile of the target child user is acquired. Based on the individual speech profile, a preset acoustic recognition model is fine-tuned to obtain a target acoustic recognition model. According to the target acoustic recognition model, speech recognition processing is performed on the speech data to be recognized to obtain the speech recognition result of the speech data to be recognized. This invention solves the problem that existing speech recognition methods have difficulty adapting to children's pronunciation patterns in children's speech scenarios, resulting in a high recognition error rate.

[0071] It is understood that in the specific implementation of this application, data such as voice data, profile data, and user data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required. Furthermore, the collection, use, and processing of related data, as well as the training, deployment, and invocation of algorithm models, must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0072] Optionally, in the step of obtaining the speech data to be recognized from the target child user, the initial speech data of the target child user can be obtained; the initial speech data can be normalized by power and subjected to spectral reduction processing to obtain preprocessed speech data; and the preprocessed speech data can be subjected to adaptive frame adjustment to obtain the speech data to be recognized.

[0073] In this embodiment of the invention, the aforementioned initial voice data can be voice data collected in real time by the voice acquisition device of the smart device when the target child user interacts with the smart device. For example, it can be voice data with unclear syllables, unstable speech rate, irregular pauses, and incomplete semantic expression.

[0074] The aforementioned power normalization can be a process of adjusting the power of the initial speech data of the target child user so that the power of all speech signals is within the same range. Power normalization aims to adjust the power of the speech signal to a uniform reference level to reduce fluctuations caused by differences in acquisition conditions (such as microphone sensitivity, user distance, or environmental noise), thereby improving the stability and accuracy of subsequent processing.

[0075] The aforementioned spectral subtraction processing can be a process of separating speech and noise from the initial speech data through frequency domain operations, and recovering the spectrum of clean speech by subtracting the estimated noise spectrum from the spectrum of the noisy speech, thereby reducing the influence of background noise. Alternatively, spectral subtraction can be used to perform spectral subtraction on the initial speech data. Spectral subtraction is a speech denoising method that recovers a clean speech spectrum by subtracting the estimated noise spectrum from the noise spectrum.

[0076] The aforementioned preprocessed speech data can be speech data obtained by power normalization and spectral subtraction of the initial speech data.

[0077] Specifically, the initial speech data can be denoted as x(t), and after power normalization and spectral subtraction, preprocessed speech data can be obtained:

[0078]

[0079] in: For preprocessing speech data; X(f) is the speech spectrum; N(f) is the estimated noise spectrum; α is the noise suppression coefficient (usually taken as 0.9–1.2); θ(f) is the phase; It is the inverse Fourier transform operator.

[0080] It should be noted that by performing power normalization and spectral subtraction on the initial speech data, environmental noise can be effectively eliminated, thereby improving the signal-to-noise ratio of subsequent feature extraction.

[0081] The aforementioned adaptive frame adjustment is a process that automatically adjusts the frame length and frame shift of the speech signal based on the characteristics of the pre-processed speech data. Adaptive frame adjustment can dynamically adjust the size or position of the frame (i.e., a small segment of the signal) used for analysis or processing to adapt to real-time changes in speech characteristics, thereby improving processing efficiency.

[0082] Specifically, this can be achieved by detecting the speech rate of preprocessed speech data. With base frequency The input frame length L is dynamically adjusted, while traditional speech recognition systems use a fixed frame length (typically 25 ms).

[0083]

[0084] Where L is the frame length; L0 is the standard frame length (e.g., 25ms). Reference adult base frequency (approximately 120Hz); This indicates the fundamental frequency of the preprocessed speech data; The average fundamental frequency for adults is approximately 120 Hz. This indicates the speech rate of the preprocessed speech data.

[0085] The aforementioned speech data to be recognized can be speech data obtained by adaptive frame adjustment processing of preprocessed speech data.

[0086] It should be noted that by adaptively adjusting the preprocessed speech data, the features of children's speech are automatically matched, reducing the difference between the input distribution of children's speech and that of adult models.

[0087] Optionally, in the step of obtaining the individual voice profile of the target child user, multiple voice samples of the target child user can be obtained; features can be extracted from the voice samples to obtain the voice features of the target child user, with each voice sample corresponding to one voice feature; based on the voice features and the number of voice samples, an individual voice profile of the target child user can be generated.

[0088] In this embodiment of the invention, the above-mentioned voice samples may be raw voice data of the target child user collected over a period of time (e.g., several consecutive days), and may be voice data in various forms such as short stories, nursery rhymes, and natural dialogues.

[0089] The aforementioned feature extraction can be a process of extracting representative speech features from speech samples. It can also be the extraction of key information representing speech features from the physical properties of sound waves. These physical properties include fundamental frequency, formants, and energy envelope; where fundamental frequency determines pitch (e.g., gender differentiation), formants determine timbre (e.g., vowel recognition), energy envelope determines volume, and duration and intonation patterns constitute prosodic features. Each speech sample corresponds to one speech feature. Voiceprint recognition models can be used to extract features from speech samples. These models can be based on deep learning or machine learning, such as ECAPA-TDNN or X-vector. ECAPA-TDNN (Extended Convolutional Architecture for Speaker Recognition) is an advanced voiceprint recognition model. By introducing a squeeze excitation module (SE module) and channel attention mechanism, the ECAPA-TDNN model significantly improves the accuracy and robustness of speaker recognition. The aforementioned X-vector is a speaker embedding vector based on a deep neural network for voiceprint recognition tasks. It maps variable-length speech to fixed-length vectors in an end-to-end manner to capture speaker identity features. The voiceprint recognition model can extract features robust to children and noise from speech samples.

[0090] The aforementioned speech features can be feature information extracted from the initial speech data that can represent the initial speech data. Speech features can characterize the stable features of the child's speech of the target child user, such as timbre distribution, formant structure, speech rate and rhythm patterns, etc.

[0091] The aforementioned individual voice profile can be a model or feature set that represents the individual voice characteristics of the target child user, generated based on voice features and the number of voice samples.

[0092] Specifically, to obtain an individual voice profile of the target child user, multiple voice samples can be collected over a period of time (e.g., several consecutive days), and the voice features can be aggregated by mean to obtain the individual voice profile of the target child user. The aforementioned mean aggregation can be the process of calculating the arithmetic mean of all voice features corresponding to multiple voice samples of the target child user in vector space, thereby obtaining the individual voice profile.

[0093] Speech features can be represented as:

[0094]

[0095] in, For speech sample i, For speech features (dimension d is typically 192–512).

[0096] An individual's voice profile can be represented as:

[0097]

[0098] in: An individual voice vector for the target child user k. The number of voice samples for the target child user k; Let be the speech features of the i-th speech sample.

[0099] It should be noted that multiple voice samples of the target child user can be obtained, and features can be extracted from the voice samples to obtain the voice features of the target child user. Each voice sample corresponds to a voice feature, and based on the voice features and the number of voice samples, an individual voice profile of the target child user can be generated. Through the individual voice profile, the voice characteristics of the target child user can be better understood.

[0100] Optionally, in the step of fine-tuning the preset acoustic recognition model based on the individual speech profile to obtain the target acoustic recognition model, the scaling factor and offset factor can be determined according to the individual speech profile; and the intermediate layer parameters of the acoustic recognition model can be fine-tuned based on the scaling factor and offset factor to obtain the target acoustic recognition model.

[0101] In this embodiment of the invention, the scaling factor can be used to amplify or reduce the features of the intermediate layer of the model element by element.

[0102] The aforementioned offset coefficients can be used to translate or offset the features of the intermediate layers of the model element by element.

[0103] The aforementioned acoustic recognition model can be an acoustic model built based on deep learning or machine learning, such as GMM-HMM, GMM-UBM, etc. This acoustic recognition model can extract acoustic features from speech data by simulating human auditory characteristics using MFCC (Mel-frequency cepstral coefficients) or Fbank (filter bank energy).

[0104] The aforementioned acoustic recognition model can map input speech data (such as speech waveforms) to corresponding speech unit sequences (such as phonemes or syllables). The core of the acoustic recognition model is to establish the mapping relationship between acoustic features and speech units. The acoustic recognition model captures the spectral information of speech by extracting the acoustic features of the speech signal, and uses statistical models or deep neural networks to calculate the probability of observed features under a given speech unit.

[0105] The aforementioned intermediate layer can be the core component of the acoustic recognition model responsible for gradually extracting and transforming speech representations from the original acoustic features.

[0106] The aforementioned fine-tuning can be a training and adjustment process that involves directional and dynamic adaptation of the intermediate layer parameters of the acoustic recognition model based on scaling and offset coefficients.

[0107] Understandably, individual voice profiles are stored locally on the smart device (1–3 MB) and used to dynamically adjust acoustic model layer parameters during recognition and inference. Individualized modulation adaptation can be achieved through Feature-wise Linear Modulation (FiLM) layers.

[0108]

[0109] in: Features of the modulated intermediate layer; Create an individual voice profile for the target child user k; Features for the intermediate layer of the model; Scaling factor determined for the individual voice profile of the target child user k; The generated offset coefficients are determined for the individual voice profile of the target child user k. Feature-wise Linear Modulation (FiLM) is a neural network module used to dynamically adjust the feature representations of intermediate layers based on conditional input, performing independent linear transformations on each feature channel by learning affine transformation parameters (scaling factor γ and offset β).

[0110] Furthermore, the parameters of the intermediate layers of the acoustic recognition model can be fine-tuned using the backpropagation algorithm. This fine-tuning process is iterated until the error loss is less than a preset value, or the number of iterations reaches a preset number, at which point the training process ends, yielding the target acoustic recognition model. The aforementioned backpropagation algorithm is based on gradient descent and aims to adjust network parameters to reduce the gap between the predicted output and the true value.

[0111] The aforementioned target acoustic recognition model can be obtained by fine-tuning the intermediate layer parameters of the acoustic recognition model based on scaling and offset coefficients. This target acoustic recognition model is capable of recognizing the speech data of target child users.

[0112] Optionally, the acoustic recognition model includes a Transformer layer. The child semantic recognition method can also freeze the original weights of the Transformer layer and insert a low-rank trainable matrix into the Transformer layer. When a model adjustment signal is detected, the low-rank trainable matrix is ​​trained locally using incremental data to obtain the adjusted low-rank trainable matrix. The Transformer layer is then updated based on the adjusted low-rank trainable matrix.

[0113] In this embodiment of the invention, the core of the Transformer layer is the Self-Attention mechanism. The Transformer layer generates context-sensitive representations by calculating the correlations between all positions in the input sequence, thereby capturing long-distance dependencies. The original weights of the Transformer layer can be obtained during the pre-training stage using large-scale general speech data, and can be the matrix of all learnable parameters (such as W) constituting the Transformer layer in the acoustic recognition model. Q W K W V W O The initial set of values ​​(W1, W2, etc.).

[0114] The aforementioned freezing process can prevent the weights of the Transformer layer from being updated, in order to preserve the learning capabilities of the original model.

[0115] The aforementioned low-rank trainable matrix can be a pair of small-dimensional, trainable parameter matrices (A and B) constrained by low-rank decomposition. The low-rank trainable matrix is ​​used to learn the personalized feature shifts of the target child user with extremely high parameter efficiency.

[0116] The aforementioned incremental data could be newly collected voice samples from target child users.

[0117] The aforementioned local training can be a process performed on the device, using only incremental data as the training set, and updating the parameters of the low-rank trainable matrix through the backpropagation algorithm. The frozen original weights W are not included in the update.

[0118] The adjusted low-rank trainable matrix described above can be obtained by locally training the low-rank trainable matrix using incremental data. The adjusted low-rank trainable matrix can adapt to changes in data distribution.

[0119] The above update can be a process of adjusting the Transformer layer using an adjusted low-rank trainable matrix to improve its performance and accuracy.

[0120] Specifically, by inserting a low-rank trainable matrix into the Transformer layer and training the low-rank trainable matrix locally using incremental data, individualized fine-tuning can be achieved on the edge:

[0121]

[0122] in: W represents the updated weights, and W represents the original weights. Indicates the weight update amount; It is a low-rank trainable matrix (r≪d, e.g., r=8); it should be noted that only A and B are fine-tuned on the device side, and the parameter increment is about 1% of the original model.

[0123] It should be noted that when a model adjustment signal is detected, the low-rank trainable matrix can be trained locally using incremental data to obtain an adjusted low-rank trainable matrix. The Transformer layer can then be updated using the adjusted low-rank trainable matrix to improve the generalization ability of the acoustic recognition model and enable it to better adapt to changes in data distribution.

[0124] Optionally, after updating the Transformer layer based on the adjusted low-rank trainable matrix, the gradient parameters obtained from local training can be uploaded to the server so that the server can update the global model through federated learning; after receiving the global model from the server and deploying the global model locally, the updated acoustic recognition model is obtained.

[0125] In this embodiment of the invention, the gradient parameters described above can be used during local training to guide how model parameters are adjusted to minimize prediction error.

[0126] The aforementioned server can be a cloud server.

[0127] Federated learning, also known as federated machine learning, is a distributed machine learning method. Its core principle is allowing multiple participants to collaboratively train a shared machine learning model without sharing the original data. The core mechanism of federated learning is to protect data privacy and solve the data silo problem. Specifically, each participant trains the model using local data, only sending updated model information (such as gradients or parameters) to the server. The server aggregates the updated information to optimize the global model, while the original data remains on its local device or within the organization, thus avoiding the privacy risks associated with centralized data storage.

[0128] The aforementioned global model can be deployed on a cloud server, continuously iterating through privacy-preserving federated learning, incorporating the collective acoustic experiences of children's users, and serving as a deployable acoustic recognition model. The global model can be pre-trained from a large amount of general-purpose speech data and continuously iterated and optimized during service by securely aggregating gradient information uploaded from multiple devices.

[0129] The aforementioned localized deployment can be a process in which the user device, after receiving the global model from the cloud, performs a series of operations locally to integrate and optimize the global model with the personalized configurations stored on the device for the current user (such as individual voice profiles, trained low-rank matrices, etc.) to obtain an updated acoustic recognition model.

[0130] The updated acoustic recognition model described above can be the acoustic recognition model obtained after the user device has completed local deployment. The updated acoustic recognition model is able to recognize children's speech data.

[0131] Specifically, the gradient parameters obtained from local training are uploaded to the server so that the server can update the global model through federated learning. The amount of global model update can be expressed as:

[0132]

[0133] in: K represents the global model update amount; K represents the number of participating devices. Let be the gradient parameters for the k-th client; The weight is the number of samples from the k-th client.

[0134] It should be noted that updating the global model through federated learning not only protects children's voice privacy but also enables the global model to continuously absorb learning results from multiple devices, achieving cross-individual adaptation.

[0135] Optionally, in the step of performing speech recognition processing on the speech data to be recognized according to the target acoustic recognition model to obtain the speech recognition result of the speech data to be recognized, the following steps can be taken: The speech data to be recognized can be processed according to the target acoustic recognition model to obtain multiple candidate speech recognition results; the candidate speech recognition results can be scored according to the contextual semantics to obtain a score for each candidate speech recognition result; the scores of the candidate speech recognition results can be weighted by confidence to obtain a confidence-weighted score for each candidate speech recognition result; and among the multiple candidate speech recognition results after confidence-weighted processing, the candidate speech recognition result with the highest confidence-weighted score can be selected as the speech recognition result of the speech data to be recognized.

[0136] In this embodiment of the invention, the target acoustic recognition model can be obtained by fine-tuning a preset acoustic recognition model using the individual voice profile of the target child user. The target acoustic recognition model is capable of recognizing the voice data of the target child user.

[0137] The aforementioned voice data to be recognized can be voice data collected in real time by the voice acquisition device of the smart device when the target child user is communicating with the smart device. It can be voice data produced by the target child user with unclear syllables, unstable speech rate, irregular pauses, and incomplete semantic expression.

[0138] The aforementioned speech recognition processing can be a process of converting speech data to be recognized into text or related instructions based on a target acoustic recognition model.

[0139] The aforementioned candidate speech recognition results can be multiple possible text sequences or related instructions generated after speech recognition processing of the speech data to be recognized based on the target acoustic recognition model.

[0140] The aforementioned contextual semantics can be the contextual semantics of all relevant situational information related to the current speech data to be recognized. It can be obtained based on historical dialogue data, task scenarios, historical habits, and real-time environmental information.

[0141] The scoring described above can be a process of scoring candidate speech recognition results based on the contextual semantics of those results. Alternatively, it can be achieved using a Transformer-based semantic re-scoring model. This model utilizes the Transformer architecture to re-evaluate and optimize prediction results in semantic segmentation tasks. The core idea of ​​this model is to capture global dependencies between pixels through a self-attention mechanism, thereby correcting or improving the accuracy of the initial segmentation results.

[0142] The scores of the aforementioned candidate speech recognition results can be obtained by scoring the candidate speech recognition results based on the contextual semantics of the candidate speech recognition results.

[0143] Specifically, the candidate speech recognition results can be scored based on their contextual semantics, resulting in a score that can be expressed as:

[0144]

[0145] in: For candidate speech recognition results The score; For candidate speech recognition results C represents the semantic features of the candidate speech recognition result; C represents the contextual semantics, such as the previous round of dialogue, topic information, and contextual information. It is understandable that the higher the score of the candidate speech recognition result, the more semantically and contextually reasonable the candidate speech recognition result is.

[0146] The aforementioned confidence weighting process can be a process of assigning a corresponding confidence score to each candidate speech recognition result based on the score of each candidate speech recognition result.

[0147] The aforementioned confidence-weighted score can be obtained by weighting the scores of candidate speech recognition results with confidence.

[0148] Furthermore, among the multiple candidate speech recognition results after confidence-weighted processing, the candidate speech recognition result with the highest confidence-weighted score can be selected as the speech recognition result of the speech data to be recognized.

[0149] The aforementioned speech recognition result can be the candidate speech recognition result with the highest confidence weight score selected from multiple candidate speech recognition results after confidence weighting processing, or it can be the text or related instructions that best match the speech data to be recognized.

[0150] In this embodiment of the invention, the invention uses the individual voice profile of the target child user to fine-tune a preset acoustic recognition model to obtain a target acoustic recognition model. Based on the target acoustic recognition model, the speech data of the target child user to be recognized is processed for speech recognition, thereby obtaining the speech recognition result of the speech data to be recognized. This can improve the efficiency and accuracy of voice interaction and solve the problem that existing speech recognition methods are difficult to adapt to children's pronunciation patterns in children's voice scenarios, resulting in a high recognition error rate.

[0151] like Figure 2 As shown, an embodiment of the present invention provides a child voice recognition device, which includes:

[0152] The acquisition module 201 is used to acquire the voice data to be identified of the target child user, and to acquire the individual voice profile of the target child user;

[0153] The first processing module 202 is used to fine-tune the preset acoustic recognition model based on the individual voice profile to obtain the target acoustic recognition model.

[0154] The second processing module 203 is used to perform speech recognition processing on the speech data to be recognized according to the target acoustic recognition model, so as to obtain the speech recognition result of the speech data to be recognized.

[0155] Optionally, the acquisition module 201 is further configured to acquire the initial speech data of the target child user; perform power normalization and spectral reduction processing on the initial speech data to obtain preprocessed speech data; and perform adaptive frame adjustment on the preprocessed speech data to obtain speech data to be recognized.

[0156] Optionally, the acquisition module 201 is further configured to acquire multiple voice samples of the target child user; extract features from the voice samples to obtain the voice features of the target child user, with each voice sample corresponding to one voice feature; and generate an individual voice profile of the target child user based on the voice features and the number of voice samples.

[0157] Optionally, the first processing module 202 is further configured to determine scaling coefficients and offset coefficients based on the individual speech profile; and to fine-tune the intermediate layer parameters of the acoustic recognition model based on the scaling coefficients and the offset coefficients to obtain the target acoustic recognition model.

[0158] Optionally, the apparatus is further configured to freeze the original weights of the Transformer layer and insert a low-rank trainable matrix into the Transformer layer; when a model adjustment signal is detected, the low-rank trainable matrix is ​​trained locally using incremental data to obtain an adjusted low-rank trainable matrix; and the Transformer layer is updated based on the adjusted low-rank trainable matrix.

[0159] Optionally, the device is further configured to upload the gradient parameters obtained from local training to a server so that the server can update the global model through federated learning; receive the global model issued by the server, and after deploying the global model locally, obtain the updated acoustic recognition model.

[0160] Optionally, the second processing module 203 is further configured to perform speech recognition processing on the speech data to be recognized according to the target acoustic recognition model to obtain multiple candidate speech recognition results; score the candidate speech recognition results according to the contextual semantics of the candidate speech recognition results to obtain a score for each candidate speech recognition result; perform confidence weighting processing on the scores of the candidate speech recognition results to obtain a confidence weighted score for multiple candidate speech recognition results, with each candidate speech recognition result corresponding to a confidence weighted score; and select the candidate speech recognition result with the highest confidence weighted score from the multiple candidate speech recognition results after confidence weighting processing as the speech recognition result of the speech data to be recognized.

[0161] like Figure 3 As shown, this embodiment of the invention also provides an electronic device, including a processor, which can execute any of the above-described child speech recognition methods.

[0162] Specifically, it includes a processor 301 and a memory 302, as well as a computer program stored in the memory 302 and capable of running on the processor 301 that performs a child speech recognition method, wherein:

[0163] The processor 301 runs the calculator program containing the children's speech recognition method stored in the memory 302, and performs the following steps:

[0164] Acquire the voice data to be identified from the target child user, and acquire the individual voice profile of the target child user;

[0165] Based on the individual voice profile, the preset acoustic recognition model is fine-tuned to obtain the target acoustic recognition model;

[0166] Based on the target acoustic recognition model, the speech data to be recognized is processed for speech recognition to obtain the speech recognition result of the speech data to be recognized.

[0167] Optionally, the process of acquiring the target child user's voice data, performed by processor 301, includes:

[0168] Obtain the initial voice data of the target child user;

[0169] The initial speech data is subjected to power normalization and spectral subtraction to obtain preprocessed speech data;

[0170] Adaptive frame adjustment is performed on the preprocessed speech data to obtain the speech data to be recognized.

[0171] Optionally, the process of obtaining the individual voice profile of the target child user executed by processor 301 includes:

[0172] Obtain multiple voice samples from the target child users;

[0173] The voice samples are subjected to feature extraction to obtain the voice features of the target child user, with each voice sample corresponding to one voice feature;

[0174] Based on the speech features and the number of speech samples, an individual speech profile of the target child user is generated.

[0175] Optionally, the processor 301 performs the step of fine-tuning the preset acoustic recognition model based on the individual speech profile to obtain the target acoustic recognition model, including:

[0176] Based on the individual voice profile, determine the scaling factor and offset factor;

[0177] Based on the scaling factor and the offset factor, the intermediate layer parameters of the acoustic recognition model are fine-tuned to obtain the target acoustic recognition model.

[0178] Optionally, the acoustic recognition model includes a Transformer layer, and the method executed by the processor 301 further includes:

[0179] The original weights of the Transformer layer are frozen, and a low-rank trainable matrix is ​​inserted into the Transformer layer.

[0180] When a model adjustment signal is detected, the low-rank trainable matrix is ​​trained locally using incremental data to obtain the adjusted low-rank trainable matrix.

[0181] The Transformer layer is updated based on the adjusted low-rank trainable matrix.

[0182] Optionally, after updating the Transformer layer based on the adjusted low-rank trainable matrix, the method executed by the processor 301 further includes:

[0183] The gradient parameters obtained from local training are uploaded to the server so that the server can update the global model through federated learning.

[0184] The system receives the global model from the server, deploys the global model locally, and then obtains the updated acoustic recognition model.

[0185] Optionally, the step of processor 301 performing speech recognition processing on the speech data to be recognized according to the target acoustic recognition model to obtain the speech recognition result of the speech data to be recognized includes:

[0186] Based on the target acoustic recognition model, the speech data to be recognized is processed for speech recognition to obtain multiple candidate speech recognition results;

[0187] Based on the contextual semantics of the candidate speech recognition results, the candidate speech recognition results are scored to obtain a score for each candidate speech recognition result.

[0188] The scores of the candidate speech recognition results are subjected to confidence weighting to obtain multiple confidence weighted scores for the candidate speech recognition results, with each candidate speech recognition result corresponding to one confidence weighted score;

[0189] Among the multiple candidate speech recognition results after confidence weighting, the candidate speech recognition result with the highest confidence weighting score is selected as the speech recognition result of the speech data to be recognized.

[0190] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the child speech recognition method provided in this invention and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0191] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method of child speech recognition, characterized by, The method comprises the following steps: acquiring voice data to be recognized of a target child user, and acquiring an individual voice portrait of the target child user; based on the individual voice portrait, fine-tuning a preset acoustic recognition model to obtain a target acoustic recognition model; performing voice recognition processing on the voice data to be recognized according to the target acoustic recognition model to obtain a voice recognition result of the voice data to be recognized.

2. The child voice recognition method of claim 1, wherein, The acquisition of the voice data to be recognized of the target child user comprises: acquiring initial voice data of the target child user; performing power normalization and spectrum reduction processing on the initial voice data to obtain preprocessed voice data; performing adaptive frame adjustment on the preprocessed voice data to obtain voice data to be recognized.

3. The child voice recognition method of claim 1, wherein, The acquisition of the individual voice portrait of the target child user comprises: acquiring a plurality of voice samples of the target child user; performing feature extraction on the voice samples to obtain voice features of the target child user, each voice sample corresponding to a voice feature; based on the voice features and the number of voice samples, generating an individual voice portrait of the target child user.

4. The child voice recognition method of claim 1, wherein, The fine-tuning of the preset acoustic recognition model based on the individual voice portrait to obtain a target acoustic recognition model comprises: determining a scaling coefficient and an offset coefficient according to the individual voice portrait; based on the scaling coefficient and the offset coefficient, fine-tuning intermediate layer parameters of the acoustic recognition model to obtain a target acoustic recognition model.

5. The child voice recognition method of claim 4, wherein, The acoustic recognition model comprises a Transformer layer, and the method further comprises: performing freezing processing on original weights of the Transformer layer, and inserting a low-rank trainable matrix in the Transformer layer; when a model adjustment signal is detected, performing local training on the low-rank trainable matrix through incremental data to obtain an adjusted low-rank trainable matrix; updating the Transformer layer based on the adjusted low-rank trainable matrix.

6. The child voice recognition method of claim 5, wherein, After the updating of the Transformer layer based on the adjusted low-rank trainable matrix, the method further comprises: uploading gradient parameters obtained through local training to a server to enable the server to update a global model through federated learning; receiving a global model issued by the server, and performing local deployment on the global model to obtain an updated acoustic recognition model.

7. The child voice recognition method of any one of claims 1-6, wherein, The voice recognition processing on the voice data to be recognized according to the target acoustic recognition model to obtain a voice recognition result of the voice data to be recognized comprises: performing voice recognition processing on the voice data to be recognized according to the target acoustic recognition model to obtain a plurality of candidate voice recognition results; scoring the candidate voice recognition results according to context semantics of the candidate voice recognition results to obtain scores of the candidate voice recognition results, each candidate voice recognition result corresponding to a score; The scores of the candidate speech recognition results are subjected to confidence weighting processing to obtain confidence weighted scores of the candidate speech recognition results, each of the candidate speech recognition results corresponding to one of the confidence weighted scores; Among the candidate speech recognition results after the confidence weighting processing, the candidate speech recognition result with the highest confidence weighted score is selected as the speech recognition result of the to-be-recognized speech data.

8. A child speech recognition apparatus, characterized by The child speech recognition device comprises: An acquisition module configured to acquire to-be-recognized speech data of a target child user and acquire an individual speech portrait of the target child user; A first processing module configured to fine-tune a preset acoustic recognition model based on the individual speech portrait to obtain a target acoustic recognition model; A second processing module configured to perform speech recognition processing on the to-be-recognized speech data according to the target acoustic recognition model to obtain a speech recognition result of the to-be-recognized speech data.

9. An electronic device, comprising: comprise: A memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the child speech recognition method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium and executable on the processor to implement the steps in the child speech recognition method according to any one of claims 1 to 7.

Images