An interactive control method of an intellectual toy based on voice recognition

Abstract

The application relates to the technical field of voice data processing and interactive control, and discloses an interactive control method for an educational toy based on voice recognition, which comprises the following steps: locally processing user voice at an edge end to generate concise voice text, matching a first interactive instruction from a local vocabulary, uploading to a cloud interactive server for semantic analysis if there is no matching item, matching a second interactive instruction from a cloud instruction library, and the educational toy executing an interactive task in response to the first and second interactive instructions, wherein, in the execution process, the edge end identifies a semantic emotion label of the user, generates a third interactive instruction when the semantic emotion label indicates that the user is in a negative or attention deviation state, and sends prompt information to a parent terminal if the third interactive instruction is triggered for multiple times in succession. The application realizes edge-cloud collaborative hierarchical interaction of the educational toy, and takes into account response speed, infant voice adaptation, emotion perception intervention and personalized learning.

Description

Technical Field

[0001] This invention relates to the field of voice data processing and interactive control technology, and in particular to an interactive control method for educational toys based on voice recognition. Background Technology

[0002] With the rapid development of artificial intelligence and voice recognition technology, educational toys are gradually evolving from traditional mechanical passive interaction to intelligent interaction. Educational toys with voice interaction functions can engage in natural language dialogue with children and perform corresponding interactive tasks according to children's instructions. They play an important role in children's early education, language development, and emotional companionship. Nowadays, parents have higher requirements for the intelligence, naturalness of interaction, and emotional companionship capabilities of educational toys. They expect toys to not only respond to instructions but also sense children's emotional state and provide appropriate feedback.

[0003] Currently, voice-interactive educational toys on the market mainly adopt two technical solutions: one is to upload the collected user voice to a cloud server for voice recognition and semantic analysis, and then return control commands; the other is to complete voice recognition and command matching on the chip built into the toy. Among them, the cloud solution has high response latency and strong network dependence, and cannot work properly in environments with poor network signal or no network. The edge solution is limited by the computing resources and storage capacity of edge devices, and has limited voice recognition accuracy and semantic understanding capabilities, making it difficult to handle complex interactive tasks.

[0004] Most existing educational toys have passive interaction mechanisms, meaning they only execute user commands and lack the ability to perceive the user's emotional state and intervene proactively. When children experience negative emotions or become distracted, the toys cannot recognize these states and provide comforting or attention-grabbing feedback, leading to interrupted interaction and a decreased user experience.

[0005] In addition, existing solutions generally adopt general speech recognition models based on standard pronunciation, which are not good at adapting to language patterns that are unique to young children, such as non-standard pronunciation, reduplicated words, and simplified sentence structures. As a result, the speech recognition rate is low and the interactive experience is poor. At the same time, the vocabulary and speech recognition models of existing toys cannot be dynamically optimized according to the user's individual pronunciation habits and interaction history. Even if the user uses the same toy for a long time, it is difficult to improve the user experience, and there is a lack of personalized interactive adaptation mechanism design. Summary of the Invention

[0006] The purpose of this invention is to propose an interactive control method for educational toys based on speech recognition in order to solve the problems in the prior art.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: an interactive control method for educational toys based on speech recognition, comprising the following steps: S1. Real-time acquisition of user voice, local preprocessing at the edge to generate concise voice text, matching the corresponding first interactive command from the local dictionary at the edge, and uploading the concise voice text to the cloud interaction server if there is no matching item in the local dictionary. S2, after receiving the concise voice text, the cloud interaction server performs semantic parsing through a semantic recognition model to obtain the parsing result, and matches the corresponding second interactive instruction from the cloud instruction library. The parsing result includes a structured semantic object and its confidence score. S3, the educational toy responds to the first and second interactive instructions to perform an interactive task. During the execution of the interactive task, the acoustic feature parameters of the user's voice are extracted by the emotion recognition model at the edge and a semantic emotion label is output. When the semantic emotion label indicates that the user's emotion is negative or attention is deviated, a third interactive instruction is generated. When the third interactive instruction is triggered more than a preset number of times, a prompt message is sent to the parent terminal.

[0008] The beneficial effects of the technical solution provided by this invention include at least the following: This invention provides a voice recognition-based interactive control method for educational toys. Through a hierarchical interactive architecture of edge-cloud collaboration, simple commands are responded to locally at the edge, while complex commands are uploaded to the cloud for response. At the same time, the edge has a built-in lightweight voice recognition network and local dictionary, enabling educational toys to operate normally in offline environments with poor network signal or no network, and respond to users' common commands. This effectively solves the problems of high latency and offline failure in pure cloud solutions and the resource limitations of pure edge solutions, significantly improving the scene adaptability of educational toys and the continuity of user experience.

[0009] This invention significantly improves speech recognition accuracy for young children with non-standard pronunciation and non-standard expression by constructing a dedicated lexicon for their language patterns and employing a fuzzy semantic matching algorithm based on edit distance and synonym expansion. At the same time, through a cloud-based global collaborative update mechanism and an edge-based personalized weight adjustment mechanism, a two-layer lexicon optimization system is formed, enabling educational toys to continuously adapt to the individual pronunciation habits of users, and the speech recognition accuracy increases with the number of uses.

[0010] This invention deploys a lightweight emotion classification network at the edge, which extracts acoustic feature parameters such as fundamental frequency, short-term energy, speech rate, and formant frequency of user speech in real time to identify three emotional states of the user: "positive," "negative," and "attention deviance." Based on this, differentiated interactive instructions are generated. At the same time, through continuous trigger counting and parent prompting mechanisms, abnormal states of children are reported in a timely manner, effectively filling the technological gap in existing educational toys in terms of emotion perception and proactive companionship. Attached Figure Description

[0011] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a method provided in an embodiment of the present invention. Detailed Implementation

[0013] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a voice recognition-based interactive control method for educational toys proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0014] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0015] The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0016] The following description, in conjunction with the accompanying drawings, details a specific scheme for an interactive control method for educational toys based on speech recognition provided by the present invention.

[0017] Please see Figure 1 The diagram illustrates a flowchart of an interactive control method for an educational toy based on speech recognition, according to an embodiment of the present invention. The method includes the following steps: S1 collects user voice in real time and performs local preprocessing at the edge to generate concise voice text. Based on the concise voice text, it matches the corresponding first interactive command from the local dictionary at the edge. If there is no matching item in the local dictionary, the concise voice text is uploaded to the cloud interaction server. S2, after receiving the concise voice text, the cloud interaction server performs semantic parsing through the semantic recognition model to obtain the parsing result, and matches the corresponding second interactive command from the cloud command library. The parsing result includes the structured semantic object and its confidence score. S3, the educational toy responds to the first and second interactive commands to perform interactive tasks. During the execution of interactive tasks, the acoustic feature parameters of the user's voice are extracted at the edge through the emotion recognition model and semantic emotion labels are output. When the semantic emotion label indicates that the user's emotion is negative or attention is deviated, a third interactive command is generated. When the third interactive command is triggered more than a preset number of times, a prompt message is sent to the parent terminal.

[0018] In one embodiment of the present invention, the steps of real-time acquisition of user voice and local preprocessing at the edge to generate concise speech-to-text include: Real-time acquisition of user voice, endpoint detection of the acquired user voice to identify the start and end points of valid voice segments, and extraction of valid voice segments; Noise suppression and pre-emphasis filtering are applied to the effective speech segments to obtain the denoised speech signal; The noise-reduced speech signal is input into a lightweight speech recognition network deployed at the edge, which outputs the corresponding pinyin sequence and generates concise speech text based on the combination of the pinyin sequence. The noise-reduced speech signal and concise speech text are cached in local storage units at the edge.

[0019] It should be noted that in this embodiment, a dual-threshold detection algorithm based on short-time energy and zero-crossing rate is used for endpoint detection. The high threshold is used to detect the start point of the speech signal, and the low threshold is used to detect the end point of the speech signal. When the short-time energy or zero-crossing rate of multiple consecutive frames exceeds the high threshold, it is determined as the start of a valid speech segment. When the short-time energy and zero-crossing rate of multiple consecutive frames are both lower than the low threshold, it is determined as the end of a valid speech segment. Subsequently, the minimum mean square error (MMSE) estimation algorithm is used to suppress noise in the valid speech segment.

[0020] In this embodiment, the lightweight speech recognition network is preferably a temporal convolutional network or a recurrent neural network based on depthwise separable convolutions, which should be able to run in real time on the microcontroller of the edge device.

[0021] In this embodiment, the noise-reduced speech signal and concise speech text are cached in the local storage unit. This aims to provide a data source that can be directly accessed for subsequent steps, avoid repeated processing of the original user speech, and reduce the computational overhead at the edge.

[0022] As one embodiment of the present invention, the step of matching the corresponding first interactive command in the local dictionary at the edge based on concise speech text includes: The concise speech text is matched with the preset entries in the local dictionary using fuzzy semantic matching based on edit distance and synonym expansion to obtain the semantic similarity score between the concise speech text and each entry. When the semantic similarity score of at least one term exceeds the preset similarity threshold, the match is determined to be successful, and the pre-stored instruction associated with the term with the highest semantic similarity score is read from the local dictionary as the first interaction instruction; When the semantic similarity scores of all terms are lower than the preset similarity threshold, the matching is deemed to have failed, the concise voice text is marked as "not matched", and uploaded to the cloud interactive server.

[0023] It should be noted that, in this embodiment, the editing distance-based matching algorithm preferably uses the Levenstein distance algorithm, which is used to calculate the minimum number of insertion, deletion, and replacement operations between the concise speech text and the word entry, i.e., the editing distance. The calculated editing distance is then normalized to a similarity score between 0 and 1, where the normalization formula is: Similarity score = 1 - edit distance / max(length of concise speech text, length of entry); When the edit distance is 0, the similarity score is 1, indicating that the concise speech text completely matches the term.

[0024] In this embodiment, the synonym expansion is based on a dedicated lexicon for children's language patterns. This dedicated lexicon pre-stores the mapping relationship between standard words and the common synonym variants of children corresponding to the standard words. When performing fuzzy semantic matching, if a word in the concise speech text matches a common synonym variant of children in the dedicated lexicon, it is mapped to the standard word and then participates in the similarity calculation to improve the recognition accuracy of children's pronunciation variations and reduplicated word expressions.

[0025] In this embodiment, the preset similarity threshold is preferably in the range of 0.6 to 0.9. The higher the threshold, the higher the matching accuracy, but the recall rate will be reduced accordingly. The lower the threshold, the higher the matching success rate, but the probability of introducing false matches increases. Professionals can configure the specific value of the similarity threshold according to the size of the local dictionary and the real-time response requirements. When a match is determined to be successful, the pre-stored instruction read from the local dictionary is an instruction code that can be directly executed on the edge, without calling cloud resources. When a match is determined to be unsuccessful, the concise voice text is marked as "unmatched" and uploaded to the cloud interaction server for further semantic analysis and matching by the cloud.

[0026] In one embodiment of the present invention, the local lexicon is a dedicated lexicon for children's language patterns. The dedicated lexicon contains at least one type of entry from children's commonly used pronunciation variants, reduplicated words, and simplified sentence structures. The method also includes a step of globally and collaboratively updating the dedicated lexicon. The cloud-based interactive server receives and stores concise voice text uploaded by multiple users and corresponding matching failure records. According to the preset first update cycle, it counts the frequency of each word in the matching failure records, marks words with a frequency exceeding a preset frequency threshold as high-frequency unmatched words, compares the similarity of high-frequency unmatched words with words in a dedicated dictionary, identifies pronunciation variants or new words, and encapsulates the identified pronunciation variants or new words into word optimization data. The edge device sends a dictionary update request to the cloud interaction server according to the preset second update cycle; In response to the dictionary update request, the cloud-based interactive server sends the latest optimized dictionary data in batches to the edge devices to perform incremental update operations on the dedicated dictionary.

[0027] It should be noted that pronunciation variants refer to words that are similar to the standard pronunciation but spelled differently when spoken by young children before their speech organs are fully developed. Reduplicated words refer to repetitive expressions commonly found in young children's language. Simplified sentence structures refer to short sentence structures in young children's language that omit grammatical components.

[0028] In this embodiment, the specific implementation process for counting the frequency of each word in the failed matching records and identifying pronunciation variants or new words is as follows: (1) The cloud interaction server performs word segmentation on the concise voice text in all the failed matching records received, counts the total number of times each word appears in all records, and forms a word-frequency mapping table. The preset frequency threshold can be dynamically configured according to the size of the user group, preferably between 50 and 1000 times. The lower the threshold, the more sensitive the word library is to updates; the higher the threshold, the more stable the word library quality. (2) Mark entries that appear more than a preset frequency threshold as high-frequency unmatched entries, indicating that the entries generally fail to match in the interactions of multiple users and have potential value for inclusion in the word library; (3) Subsequently, the similarity between each high-frequency unmatched word and the existing words in the dedicated word library is calculated one by one using the edit distance algorithm based on pinyin or the pronunciation similarity algorithm based on the acoustic confusion matrix. The similarity score between 0 and 1 is output. When the similarity score between at least one existing word and the high-frequency unmatched word exceeds the preset similarity threshold, the high-frequency unmatched word is identified as a pronunciation variant of the existing word and a mapping relationship between it and the existing word is established. When the similarity scores between all existing words and the high-frequency unmatched words are lower than the preset similarity threshold, the high-frequency unmatched word is identified as a new word.

[0029] In this embodiment, when the identified pronunciation variant words or new terms are encapsulated into term optimization data, they are encapsulated in JSON format or a similar structured data format for edge parsing and incremental updates. For pronunciation variant words, the optimization data includes: variant word text, mapped existing terms, and initial weight values ​​(e.g., 0.6-0.8); for new terms, the optimization data includes: new term text and initial weight values ​​(e.g., 1.0).

[0030] During the global collaborative update process, the matching failure records include concise voice text uploaded by the edge device and its corresponding context information. The first update cycle is preferably configured to be between 24 hours and 7 days. The cloud server performs batch statistics based on the accumulated matching failure records. The second update cycle can be independently configured to be between 6 hours and 7 days, and is usually set to be shorter than the first update cycle to ensure that the edge device can obtain the latest optimized data from the cloud statistics in a timely manner. The incremental update operation for the dedicated dictionary only distributes the newly added or changed word data to reduce network transmission load and storage overhead at the edge device.

[0031] In one embodiment of the present invention, the steps of receiving concise voice text, performing semantic parsing through a semantic recognition model to obtain the parsing result, and outputting a second interactive command by the cloud interactive server include: After receiving concise speech text, the cloud-based interactive server performs word segmentation and part-of-speech tagging on the concise speech text through a semantic recognition model, extracting task keywords and task parameters. The semantic recognition model adopts the Transformer architecture and is pre-tuned and trained using a corpus of children's speech text. Task keywords include action verbs and interactive object nouns, and task parameters include at least one of quantity, color, and location. The semantic recognition model combines the extracted task keywords with task parameters into a structured semantic object; The matching process is based on structured semantic objects. The corresponding second interactive instruction is matched from the cloud instruction library. If the match is successful, the second interactive instruction is output. If the match fails, a preset guided restatement instruction is output, which is used to prompt the user to restate the statement through the voice module of the educational toy.

[0032] It should be noted that, in this embodiment, the core structure of the semantic recognition model includes a multi-head self-attention layer, a feedforward neural network layer, a residual connection layer, and a normalization layer, with the number of parameters controlled within 50M to balance parsing accuracy and response latency.

[0033] The construction method of the preschool speech-text corpus includes: collecting real speech data of children aged 3 to 8 interacting with educational toys, converting it into text through speech recognition, and manually annotating its word segmentation boundaries, part-of-speech tags, task keywords and task parameters. The corpus contains at least 100,000 annotated samples to ensure the adaptability of the semantic recognition model trained by fine-tuning the corpus to the language patterns of preschool children.

[0034] In this embodiment, based on the fine-tuned semantic recognition model, the maximum matching word segmentation algorithm is used to segment concise speech text, the label sequence output by the Transformer model is used for part-of-speech tagging, and the task keywords are extracted through the semantic role tagging method. The resulting structured semantic object is a standardized JSON format data object, which contains the following fields: {"action": "action verb", "object": "interactive object noun", "attributes": {"quantity": "numerical value", "color": "color value", "location": "location description"}}; The task parameters in the attributes field can be dynamically filled based on the parsing results, and missing parameters can be marked as "none" or "to be completed".

[0035] In this embodiment, the cloud-based instruction library is a pre-built instruction mapping table. Each instruction in the table corresponds to one or more structured semantic objects. When performing the matching step, a tree-based matching algorithm or a vector-based cosine similarity algorithm is used to calculate the similarity between the structured semantic object and each instruction in the table. When the matching fails, the output guidance restatement instruction is a preset voice prompt template, such as "You can say to me like this: 'Give me the red block'". It should be noted that this guidance restatement instruction is a static and general example guidance, which aims to guide the user to restate the task requirements in a way that conforms to the semantic model's parsing capabilities. In this step, it does not change with the different concise voice text of the failed match.

[0036] As one embodiment of the present invention, before performing the matching step based on the structured semantic object and matching the corresponding second interactive instruction from the cloud instruction library, a confidence assessment step is further included: The semantic recognition model calculates the confidence score of the structured semantic objects while outputting them. The confidence score is compared with a preset first confidence threshold and a second confidence threshold, wherein the first confidence threshold is higher than the second confidence threshold; If the confidence score is higher than the first confidence threshold, then the matching step is performed; If the confidence score is higher than the second confidence threshold but lower than the first confidence threshold, the voice module of the educational toy will issue an ambiguity resolution question to the user. After correcting the structured semantic object based on the user's feedback, the matching step will continue. If the confidence score is lower than the second confidence threshold, the matching step is not performed, and a preset unrecognized prompt is output.

[0037] It should be noted that in this embodiment, the confidence score is generated synchronously by the semantic recognition model when outputting the structured semantic object. That is, based on the normalized probability distribution output by the semantic recognition model, the cumulative probability values ​​of each field that makes up the structured semantic object on its output path are extracted and weighted and fused. Preferably, the maximum probability value output by the Softmax function is used as the confidence of each field, and then the confidence score of the structured semantic object composed of these fields is obtained by weighted averaging. When allocating weights, the weight of the task keywords should be higher than the weight of the task parameters.

[0038] In this embodiment, the first confidence threshold is preferably set between 0.85 and 0.95 to determine whether the parsing result is accurate enough for direct execution, and the second confidence threshold is preferably set between 0.5 and 0.7 to determine whether the parsing result is ambiguous on the basis of having reference value. Those skilled in the art can adaptively adjust the specific values ​​of the first confidence threshold and the second confidence threshold according to the dynamic feedback of the actual application scenario.

[0039] In this embodiment, the specific implementation process of the ambiguity resolution query is as follows: (1) When the confidence score is between the first threshold and the second threshold, identify one or more fields with the lowest confidence score in the structured semantic object as ambiguous fields; (2) For ambiguous fields, extract a list of possible candidate words from the cloud instruction library, and then generate targeted ambiguity resolution queries. For example, if the "interaction object noun" field is confirmed as "building block", when the confidence of the "color" field is low, ask "Do you mean the red building block or the blue building block?", and when the confidence of the "action verb" field is low, ask "Do you want to build or dismantle the building blocks?". (3) Collect user feedback, and after edge speech recognition, parse it into the candidate words selected by the user or the complete speech command re-expressed by the user, which are used to replace the original ambiguous fields and generate the corrected structured semantic object. It should be noted that if the user does not make a valid candidate word selection or rephrase within the preset time, the subsequent matching steps will not be executed. Instead, a preset unrecognized prompt instruction will be output. The unrecognized prompt instruction is a preset voice prompt template, such as "I didn't hear you clearly, can you say it again?" or "Please tell me louder", which is used to prompt the user to re-initiate the voice command. After outputting the unrecognized prompt instruction, the system returns to step S1 and triggers the edge device to re-collect the user's voice.

[0040] In one embodiment of the present invention, the step of extracting acoustic feature parameters of user speech and outputting semantic emotion labels at the edge using an emotion recognition model includes: The edge device obtains the noise-reduced audio signal of the user's voice in real time from the local storage unit; The denoised speech signal is segmented and windowed. The fundamental frequency, short-time energy and speech rate are extracted from each frame as acoustic feature parameters, and the formant frequency is calculated based on the fundamental frequency change of multiple consecutive frames. The extracted fundamental frequency, short-time energy, speech rate, and formant frequency are input into a lightweight emotion classification network deployed at the edge, and semantic emotion labels are output. The lightweight emotion classification network is pre-trained using a corpus of children's speech emotion annotations, and the semantic emotion labels are at least one of "positive", "negative", and "attention deviation".

[0041] It should be noted that in this embodiment, the frame segmentation process uses a Hamming window with a frame length of 25ms and a frame shift of 10ms, with a 15ms overlap between adjacent frames. The windowing process uses the Hamming window function w(n): w(n)=0.54-0.46cos(2πn / (N-1)); Where N is the number of sampling points corresponding to the frame length, used to suppress spectral leakage.

[0042] In this embodiment, the acoustic feature parameters extracted from the denoised speech signal include: Fundamental frequency (F0): Obtained by autocorrelation method, that is, by searching for the peak position of the autocorrelation function within the candidate fundamental frequency range; Short-time energy E: Calculated using the following formula: E = ∑[x(n)] 2 That is, the sum of squares of the audio signal sampling points x(n) in each frame is calculated; Speech rate: Calculated by counting the number of effective speech frames per unit time and dividing by a preset frame rate normalization coefficient; In addition, the formant frequencies were obtained using the Linear Predictive Coding (LPC) analysis method, which involves tracking the fundamental frequency changes over multiple consecutive frames to extract the frequency values ​​of the first formant (F1) and the second formant (F2).

[0043] In this embodiment, the lightweight emotion classification network is preferably based on a decision tree ensemble model (such as random forest or XGBoost), with the number of parameters controlled within 100k, so that it can run in real time on the microcontroller of the edge device and the single inference latency is less than 50ms.

[0044] The construction method of the corpus for annotating the speech emotions of young children is as follows: collect real speech segments of children aged 3 to 8 during their interaction with educational toys, and have human annotators label each speech segment with three emotion tags: "positive" (such as happy, excited), "negative" (such as frustrated, crying) or "attention deficit" (such as daydreaming, slow response) based on the tone, energy changes and semantic content of the speech. When selecting the corpus, a corpus containing at least 50,000 labeled samples with a balanced distribution of the sample size for each emotion category should be selected to ensure the training effect of the emotion classification network.

[0045] During the pre-training process, the cross-entropy loss function and stochastic gradient descent optimizer were used to divide the acquired children's speech emotion annotation corpus into training set, validation set and test set in a ratio of 8:1:1. Training was stopped when the accuracy of the validation set no longer improved.

[0046] The pre-trained lightweight emotion classification network performs forward computation on the input feature vector (i.e., the fundamental frequency, short-time energy, speech rate, and formant frequencies extracted in the previous steps) and outputs a three-dimensional probability distribution vector: [p_positive, p_negative, p_attentional deviation]; The label corresponding to the dimension with the highest probability value is taken as the output semantic sentiment label. Preferably, if the probability value corresponding to the dimension with the highest probability value is still lower than the preset lower limit (such as 0.6), then the label "unrecognized" is not used, and the semantic sentiment label of the previous recognition is used or a re-collection is triggered.

[0047] In one embodiment of the present invention, after outputting the semantic emotion tag, the method further includes a step of generating a third interactive instruction when the semantic emotion tag indicates that the user's emotion is negative or that their attention is diverted: When the semantic emotion label is "negative", a first-level third interaction instruction is generated to control the educational toy to perform at least one of playing soothing music or outputting a calming voice. When the semantic emotion label is "attention deviation", a second-level third interaction instruction is generated to control the educational toy to perform at least one of the following: activate dynamic light sequence, output fun voice prompts, or change the toy's action mode. When the two semantic emotion labels "negative" and "attention deviation" appear at the same time, the third interaction instruction of the first level is executed first, and the third interaction instruction of the second level is executed after the third interaction instruction of the first level is executed. After executing the third interactive instruction, the system continues to monitor the semantic emotion label of subsequent user voice through the emotion recognition model. If the semantic emotion label is still "negative" or "attention deviates" within the preset time window, an upgraded third interactive instruction is generated to perform at least one of the following actions based on the original instruction: increase volume, increase the flashing frequency of lights, or increase the amplitude of movements.

[0048] It should be noted that in this embodiment, the soothing music in the third interactive instruction of the first level is a preset track that has been verified by child psychology to have an emotional calming effect, such as alpha wave music or natural white noise (such as the sound of ocean waves or rain). Its initial volume is preferably 45-55dB and its initial playback duration is preferably 15 seconds. The corresponding soothing voice is a preset encouraging sentence template, such as "It's okay, we'll take it slowly" or "You're great, can you try again?" In the third interactive instruction of the second level, the corresponding dynamic light sequence includes alternating flashing colored LED lights, the colors of which can be selected from highly saturated colors such as red, yellow, and blue, the initial flashing frequency is preferably 1Hz, and the initial brightness is preferably 50%; the corresponding fun voice prompts are preset voice sentence templates with sound effect changes, such as "Look here, there is a little secret"; the corresponding changes to the toy's action mode include controlling the toy to produce at least one of the following actions: swinging, rotating, emitting light, or making a sound.

[0049] In this embodiment, within a preset time window (e.g., 3-5 seconds), the emotion recognition model outputs the labels "negative" and "attention deviation" sequentially or simultaneously. At this time, the first-level instruction is executed first. The reason is that children's negative emotions are usually accompanied by higher psychological needs such as a desire for comfort and encouragement, while attention deviation may only be a temporary shift in attention. Prioritizing the comforting of negative emotions helps to improve the user's interaction state.

[0050] In this embodiment, the generation of the upgraded third interactive command follows a progressively increasing principle, including: For the third interactive instruction of the first level, the upgrade strategy includes: gradually increasing the volume of the corresponding soothing music from the initial 45-55dB to 65-75dB, extending the playback duration from the initial 15 seconds to 30 seconds, and increasing the number of repetitions of the corresponding reassuring voice to 3 times. For the third interactive command at the second level, the upgrade strategy includes: increasing the flashing frequency of the corresponding dynamic light sequence from 1Hz to 3Hz, increasing the brightness from 50% to 100%, increasing the repetition of the corresponding fun voice prompts to 3 times, and increasing the range of motion of the toy when changing the toy's action mode, such as increasing the swing angle from 15° to 30°.

[0051] In this embodiment, the edge device maintains a continuous trigger counter with an initial value of zero. Each time a third interactive instruction is generated (including the first level, the second level, or the upgraded third interactive instruction), the edge device determines whether the time interval between the current generation and the previous generation of the third interactive instruction is less than a preset time interval threshold (such as 30 seconds). If so, the continuous trigger counter is incremented by 1; if the time interval exceeds the time interval threshold, the continuous trigger counter is reset to zero. When the continuous trigger counter reaches a preset threshold (e.g., 3 times), the edge device sends a prompt message to the pre-set parent terminal through its associated wireless communication module. The prompt message includes: the user's current semantic emotion tag, the type of the third interactive instruction that has been executed, and the number of times the third interactive instruction has been triggered consecutively. The parent terminal can be a smartphone, tablet, or smartwatch. The prompt message is presented in the form of a push notification or an in-app message. After sending a notification to the parent's device, the edge device continues to monitor the semantic emotion tags of subsequent user voice messages. If the continuous trigger counter does not increase within a preset silence time window (e.g., 60 seconds), the count value of the continuous trigger counter is gradually decremented (e.g., decremented by 1 every 30 seconds). If the user's emotional state returns to "positive," the continuous trigger counter is reset to zero. As one embodiment of the present invention, the process of extracting acoustic feature parameters of user speech and outputting semantic emotion labels through an emotion recognition model at the edge end also includes the following correction step: Obtain the parsing results of the semantic recognition model on concise speech text, including structured semantic objects and their confidence scores; When the confidence score is lower than the preset third confidence threshold, the local dictionary is invoked, and the pronunciation similarity score between the structured semantic object and each word in the local dictionary is calculated using a fuzzy matching algorithm based on pronunciation similarity. The dictionary selects the word with the highest pronunciation similarity score as a candidate word, corrects the parsing results based on the candidate words, and generates a corrected parsing result. The voice module of the educational toy sends pronunciation correction questions to the user, including presenting the corrected analysis results and requesting the user's confirmation; The system receives user feedback on pronunciation correction inquiries. If the user confirms, the system replaces the original analysis result with the corrected result and generates a confirmation flag. If the user denies or fails to respond within a preset time, the system maintains the original analysis result and generates a denial flag.

[0052] It should be noted that in this embodiment, the third confidence threshold is preset to be between 0.5 and 0.7, and should be lower than the second confidence threshold in the aforementioned steps.

[0053] In this embodiment, the fuzzy matching algorithm based on pronunciation similarity is preferably an algorithm based on pinyin edit distance or an algorithm based on acoustic confusion matrix. The pronunciation similarity score is calculated with a value ranging from 0 to 1. If the pronunciation similarity of the word with the highest pronunciation similarity score to the structured semantic object is still lower than a preset lower limit, then automatic correction is abandoned, the original parsing result is maintained, and a denial label is generated.

[0054] The corrected parsing result is as follows: replace the fields in the original structured semantic object with the standard words corresponding to the candidate words if the confidence score is lower than the preset third confidence threshold, while keeping other fields unchanged, and generate an updated structured semantic object. For example, when the user's voice recognition is "I want to play happily", in the original structured semantic object, the interaction object noun field may be "happy" (low confidence), and the corresponding candidate word is "happy". The corrected parsing result is: replace "happy" in the structured semantic object with "happy".

[0055] The pronunciation correction questions are presented by playing preset question templates through the voice module of the educational toy, such as "Did you just say 'happy'?" or "You meant to say 'happy', right?".

[0056] The method for collecting user confirmation feedback is as follows: the user's answer is received through the voice acquisition module, such as the user answering "yes", "correct" or "no" or "incorrect", and then parsed as confirmation or denial after keyword recognition at the edge terminal.

[0057] The preset time is preferably set to 3 to 5 seconds. If no user feedback is received within the time limit, the default is to deny and a denial mark is generated.

[0058] In one embodiment of the present invention, after generating the corrected analytical result, the method further includes the following steps: Record correction information during the correction process, including structured semantic objects, candidate words, similarity scores, and user feedback on pronunciation correction queries; If the feedback is a confirmation flag, the weight value of the candidate term is increased; if the feedback is a denial flag, the weight value of the candidate term is decreased. Based on the accumulated weight values ​​of each term in the local thesaurus, personalized weight adjustment parameters are generated for the current user to adjust the matching priority of terms in the local thesaurus. Personalized weight adjustment parameters are stored in local storage units at the edge and associated with the current user's ID for use in the user's subsequent voice interactions.

[0059] It should be noted that in this embodiment, the incremental update method is used to increase or decrease the weight value of the candidate terms, including: each time the user confirms the correction, the weight value of the candidate terms increases by a preset incremental step size (e.g., +0.05 to +0.1); each time the user denies the correction, the weight value of the candidate terms decreases by a preset decrement step size (e.g., -0.03 to -0.05). The weight values ​​of each candidate term are limited to a range of 0.5 to 2.0. Preferably, the initial weight value is preset to 1.0. When the weight value reaches the upper limit of 2.0, the weight of the candidate term will not be increased when the user confirms the correction. When the weight value drops to the lower limit of 0.5, the weight of the candidate term will not be reduced when the user denies the correction. For multiple corrections of the same term, the weight value is accumulated in each correction.

[0060] In this embodiment, the personalized weight adjustment parameter is a weight mapping table based on terms, represented as a set of key-value pairs, as follows: {“Term A”: weight value a, “Term B”: weight value b, ...}; This weight mapping table is independent of the global word data in the local dictionary. It only stores weight values ​​and does not modify the original data in the local dictionary. Preferably, after establishing the personalized weight adjustment parameter, during the local dictionary matching process, the edge device first loads the personalized weight adjustment parameter, calculates the semantic similarity score between the concise speech text and each word, multiplies the score by the weight value of the corresponding word to obtain the weighted matching score, and then performs matching judgment based on the weighted matching score.

[0061] The current user ID association mechanism includes: each educational toy is locally associated with an ID at the edge, which serves as the user ID for using that educational toy.

[0062] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for interactive control of educational toys based on speech recognition, characterized in that, The method includes: S1. Real-time acquisition of user voice, local preprocessing at the edge to generate concise voice text, matching the corresponding first interactive command from the local dictionary at the edge, and uploading the concise voice text to the cloud interaction server if there is no matching item in the local dictionary. S2, after receiving the concise voice text, the cloud interaction server performs semantic parsing through a semantic recognition model to obtain the parsing result, and matches the corresponding second interactive instruction from the cloud instruction library. The parsing result includes a structured semantic object and its confidence score. S3, the educational toy responds to the first and second interactive instructions to perform an interactive task. During the execution of the interactive task, the acoustic feature parameters of the user's voice are extracted by the emotion recognition model at the edge and a semantic emotion label is output. When the semantic emotion label indicates that the user's emotion is negative or attention is deviated, a third interactive instruction is generated. When the third interactive instruction is triggered more than a preset number of times, a prompt message is sent to the parent terminal.

2. The interactive control method for educational toys based on speech recognition according to claim 1, characterized in that: The steps involved in real-time acquisition of user voice data, local preprocessing at the edge, and generation of concise speech-to-text include: Real-time acquisition of user voice, endpoint detection of the acquired user voice to identify the start and end points of valid voice segments, and extraction of the valid voice segments; The effective speech segment is subjected to noise suppression and pre-emphasis filtering to obtain a noise-reduced speech signal; The noise-reduced speech signal is input into a lightweight speech recognition network deployed at the edge, which outputs the corresponding pinyin sequence and generates the concise speech text based on the combination of the pinyin sequence. The noise-reduced speech signal and the concise speech text are cached in the local storage unit at the edge.

3. The interactive control method for educational toys based on speech recognition according to claim 1, characterized in that: The step of matching the corresponding first interactive command in the local dictionary at the edge based on the concise voice text includes: The concise speech text is matched with preset entries in the local dictionary using fuzzy semantic matching based on edit distance and synonym expansion to obtain the semantic similarity score between the concise speech text and each entry. When the semantic similarity score of at least one term exceeds a preset similarity threshold, the match is determined to be successful, and the pre-stored instruction associated with the term with the highest semantic similarity score is read from the local thesaurus as the first interaction instruction; When the semantic similarity scores of all terms are lower than the preset similarity threshold, the matching is deemed to have failed, the concise voice text is marked as "not matched", and uploaded to the cloud interactive server.

4. The interactive control method for educational toys based on speech recognition according to claim 1, characterized in that: The local lexicon is a dedicated lexicon for early childhood language patterns. This dedicated lexicon contains at least one type of entry from commonly used pronunciation variations, reduplicated words, and simplified sentence structures. The method also includes a step of globally and collaboratively updating this dedicated lexicon. The cloud-based interactive server receives and stores the concise voice text uploaded by multiple users and the corresponding matching failure records. According to a preset first update cycle, it counts the frequency of occurrence of each word in the matching failure records, marks words with a frequency exceeding a preset frequency threshold as high-frequency unmatched words, compares the similarity of the high-frequency unmatched words with the words in the dedicated dictionary, identifies pronunciation variant words or new words, and encapsulates the identified pronunciation variant words or new words into word optimization data. The edge device sends a dictionary update request to the cloud interaction server according to a preset second update cycle; In response to the dictionary update request, the cloud-based interactive server sends the latest optimized dictionary data in batches to the edge device to perform incremental update operations on the dedicated dictionary.

5. The interactive control method for educational toys based on speech recognition according to claim 1, characterized in that: The steps of receiving the concise voice text, performing semantic parsing through a semantic recognition model to obtain the parsing result, and outputting the second interactive command by the cloud-based interactive server include: After receiving the concise voice text, the cloud-based interactive server performs word segmentation and part-of-speech tagging on the concise voice text through the semantic recognition model, and extracts task keywords and task parameters. The semantic recognition model adopts the Transformer architecture and is pre-trained with a corpus of children's voice text. The task keywords include action verbs and interactive object nouns, and the task parameters include at least one of quantity, color, and location. The semantic recognition model combines the extracted task keywords and task parameters into a structured semantic object; The matching step is performed based on the structured semantic object. The corresponding second interactive instruction is matched from the cloud instruction library. If the match is successful, the second interactive instruction is output. If the match fails, a preset guided restatement instruction is output, which is used to prompt the user to restate the statement through the voice module of the educational toy.

6. The interactive control method for educational toys based on speech recognition according to claim 5, characterized in that: The process of performing the matching step based on the structured semantic object, before matching the corresponding second interactive instruction from the cloud instruction library, further includes a confidence assessment step: The semantic recognition model outputs the structured semantic object while calculating the confidence score of the structured semantic object; The confidence score is compared with a preset first confidence threshold and a second confidence threshold, wherein the first confidence threshold is higher than the second confidence threshold; If the confidence score is higher than the first confidence threshold, then the matching step is performed; If the confidence score is higher than the second confidence threshold but lower than the first confidence threshold, the voice module of the educational toy will issue an ambiguity resolution question to the user. After correcting the structured semantic object based on the user's feedback, the matching step will continue. If the confidence score is lower than the second confidence threshold, the matching step is not performed, and a preset unrecognized prompt instruction is output.

7. The interactive control method for educational toys based on speech recognition according to claim 2, characterized in that: The steps of extracting acoustic feature parameters of the user's speech and outputting semantic emotion labels at the edge include: The edge device acquires the noise-reduced audio signal of the user's voice in real time from the local storage unit; The denoised speech signal is segmented and windowed. The fundamental frequency, short-time energy and speech rate are extracted from each frame as acoustic feature parameters, and the formant frequency is calculated based on the fundamental frequency change of multiple consecutive frames. The extracted fundamental frequency, short-time energy, speech rate, and formant frequency are input into a lightweight emotion classification network deployed at the edge, and the semantic emotion label is output. The lightweight emotion classification network is pre-trained using a corpus of children's speech emotion annotations, and the semantic emotion label is at least one of "positive", "negative", and "attention deviation".

8. The interactive control method for educational toys based on speech recognition according to claim 7, characterized in that: After outputting the semantic emotion tag, the process also includes a step of generating a third interactive instruction when the semantic emotion tag indicates that the user's emotion is negative or their attention is diverted. When the semantic emotion label is "negative", a first-level third interaction instruction is generated to control the educational toy to perform at least one of playing soothing music or outputting comforting voice. When the semantic emotion label is "attention deviation", a second-level third interactive instruction is generated to control the educational toy to perform at least one of the following: activate dynamic light sequence, output fun voice prompts, or change the toy's action mode. When the two semantic emotion labels "negative" and "attention deviation" appear at the same time, the third interaction instruction of the first level is executed first, and the third interaction instruction of the second level is executed after the third interaction instruction of the first level is executed. After executing the third interactive instruction, the semantic emotion label of the user's speech is monitored through the emotion recognition model. If the semantic emotion label is still "negative" or "attention deviates" within a preset time window, an upgraded third interactive instruction is generated to perform at least one of the following on the basis of the original instruction: increase the volume, increase the flashing frequency of the lights, or increase the amplitude of the movements.

9. The interactive control method for educational toys based on speech recognition according to claim 1, characterized in that: The process of extracting acoustic feature parameters of the user's speech and outputting semantic emotion labels through an emotion recognition model at the edge also includes the following correction steps: Obtain the parsing results of the semantic recognition model on the concise speech text, including the structured semantic objects and their confidence scores; When the confidence score is lower than the preset third confidence threshold, the local dictionary is invoked, and the pronunciation similarity score between the structured semantic object and each word in the local dictionary is calculated using a fuzzy matching algorithm based on pronunciation similarity. The word with the highest pronunciation similarity score is selected from the local dictionary as a candidate word, and the parsing result is corrected based on the candidate word to generate a corrected parsing result; The voice module of the educational toy issues a pronunciation correction query to the user, including presenting the corrected parsing result and requesting the user's confirmation; The system receives feedback from the user regarding the pronunciation correction query. If the user confirms, the system replaces the original analysis result with the corrected analysis result and generates a confirmation flag. If the user denies or fails to respond within a preset time, the system maintains the original analysis result and generates a denial flag.

10. The interactive control method for educational toys based on speech recognition according to claim 9, characterized in that: After generating the corrected analytical results, the process also includes the following steps: Record correction information during the correction process, including the structured semantic object, the candidate words, the similarity score, and the user's feedback on the pronunciation correction query; If the feedback is a confirmation flag, the weight value of the candidate term is increased; if the feedback is a denial flag, the weight value of the candidate term is decreased. Based on the accumulated weight values ​​of each term in the local thesaurus, personalized weight adjustment parameters are generated for the current user to adjust the matching priority of terms in the local thesaurus. The personalized weight adjustment parameters are stored in the local storage unit at the edge and associated with the current user's ID for use in the user's subsequent voice interactions.

Images