Speech processing method of early education robot and early education robot

Abstract

The invention discloses a voice processing method of an early education robot and the early education robot, and relates to the technical field of voice processing, and the method comprises the steps: receiving a voice instruction inputted by a user for recognition, and when the voice instruction is recognized as a teaching instruction, displaying the teaching content in the teaching instruction to the user; after the teaching content is displayed, receiving feedback voice of a user and collecting a feedback image of the user; the feedback voice and the feedback image are subjected to content consistency verification, after the consistency verification is passed, whether the feedback voice and the feedback image conform to the teaching content is judged, an early education processing result is obtained and displayed to the user, and the early education processing result comprises feedback correctness or feedback error. The technical problems that in the prior art, an early education robot is large in voice recognition computing power consumption and poor in early education effect are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of speech processing, and in particular to a speech processing method of an early childhood education robot and the early childhood education robot. Background Art

[0002] Early childhood education robots are gradually becoming an important tool in children's education. These robots usually interact with children through voice recognition technology to present early childhood education content. However, most existing early childhood education robots use voice recognition as the main interaction method, and their technical complexity and computing power consumption problems are gradually emerging.

[0003] Current early childhood education robots need to recognize the voices of adults and children, which consumes a lot of computing power. They use voice recognition as the main method of interaction and do not combine children's movements and behaviors, resulting in poor early childhood education effects. Summary of the invention

[0004] The present application provides a speech processing method for an early childhood education robot, which is used to solve the technical problems in the prior art of high consumption of computing power for speech recognition of early childhood education robots and poor early childhood education effects.

[0005] In view of the above problems, the present application provides a speech processing method for an early childhood education robot and an early childhood education robot.

[0006] In a first aspect, the present application provides a speech processing method for an early childhood education robot, the method comprising: receiving a speech instruction input by a user for recognition, and when the speech instruction is recognized as a teaching instruction, displaying the teaching content in the teaching instruction to the user; After the teaching content is presented, receiving user feedback voice and collecting user feedback images; The feedback voice and feedback image are checked for content consistency. After the consistency check is passed, it is determined whether they are consistent with the teaching content, and the early education processing result is obtained and displayed to the user, wherein the early education processing result includes correct feedback or incorrect feedback.

[0007] In the second aspect, the present application provides an early childhood education robot, including: A voice instruction parsing module is used to receive and recognize the voice instruction input by the user, and when it is recognized as a teaching instruction, display the teaching content in the teaching instruction to the user; A feedback information collection module, used to receive user feedback voice and collect user feedback images after the teaching content is presented; The feedback information processing module is used to perform content consistency verification on the feedback voice and feedback image. After the consistency verification is passed, it is determined whether they are consistent with the teaching content, and the early education processing result is obtained and displayed to the user, wherein the early education processing result includes correct feedback or incorrect feedback.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages: The present application proposes a speech processing method for an early childhood education robot and an early childhood education robot, which divides the input speech into adult speech and child speech through speech category recognition, and further combines speech content recognition, significantly simplifying the speech processing process, while improving the accuracy of command recognition. Compared with traditional speech recognition methods, the present invention can quickly identify teaching speech, avoid complex calculations of irrelevant speech, and reduce computing power consumption. The present application also uses the feature consistency of speech and children's action images for early childhood education and double verification, effectively improving the diversity of interaction when using early childhood education robots, and improving children's early childhood education enthusiasm and early childhood education effects. The present application achieves the technical effect of improving the diversity and teaching effect of the interaction between early childhood education robots and children, and reducing the consumption of speech recognition computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flow chart of a speech processing method for an early childhood education robot provided in an embodiment of the present application; Figure 2 A logical schematic diagram of a speech processing method for an early childhood education robot provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of the early childhood education robot provided in an embodiment of the present application; In the accompanying drawings, the components represented by the reference numerals are described as follows: Voice instruction parsing module 11, feedback information collecting module 12, feedback information processing module 13. DETAILED DESCRIPTION

[0011] The present application provides a speech processing method for an early childhood education robot and an early childhood education robot, aiming to solve the technical problems in the prior art of large consumption of computing power for speech recognition of early childhood education robots and poor early childhood education effects.

[0012] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0013] It should be noted that the terms "including" and "having" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules that are not explicitly listed or inherent to these processes, methods, products or devices.

[0014] Embodiment 1, as Figure 1 and Figure 2 As shown, the present application provides a speech processing method for an early childhood education robot, wherein the method is applied to a collaborative charging platform, the collaborative charging platform is communicatively connected with a central control unit and a plurality of wireless charging devices, and the method includes: S100: receiving a voice command input by a user, performing recognition, and when the voice command is recognized as a teaching command, displaying the teaching content in the teaching command to the user; In the embodiment of the present application, during the use of the early childhood education robot, the user's voice input is captured in real time through a voice acquisition device and converted into a digital voice signal.

[0015] Then, the collected voice signals are processed, including signal preprocessing (such as denoising and signal enhancement) and feature extraction (such as extraction of voice spectrum and acoustic features). During the recognition process, voice commands will be classified into multiple types, such as teaching, reward, power on and off, etc. Among them, the "teaching command" instructs the robot to enter the education mode, which is used to display specific teaching content.

[0016] Exemplarily, a voice category recognition algorithm is used to recognize the input voice command.

[0017] When the voice command is recognized as a teaching command, the teaching content in the teaching command, such as digital teaching content, is displayed to the user.

[0018] Step S100 in the method provided in the embodiment of the present application includes: Receiving voice commands input by a user; Performing voice category recognition on the voice instruction to obtain a voice category recognition result, wherein the voice category recognition result includes one of an adult voice and a child voice; Performing content recognition on the voice command to obtain a voice content recognition result, wherein the voice content recognition result includes one of power on, power off, teaching and rewarding; Combining the voice category recognition result and the voice content recognition result to obtain a voice command recognition result; When the voice instruction recognition result includes adult voice and teaching, the voice instruction is determined to be a teaching instruction, and the teaching content in the teaching instruction is displayed to the user.

[0019] In the embodiment of the present application, the user's voice input signal is captured in real time through a voice acquisition device such as a microphone. The voice signal is preliminarily processed, including denoising, volume equalization and digital processing, and the processing result is used as the voice command to be recognized to provide high-quality data support for subsequent analysis.

[0020] Furthermore, the voice commands are identified and classified to identify whether the source of the voice is an adult or a child. If the content of the voice command is directly identified, since the voice features of adults and children are quite different, a model that can simultaneously recognize the voices of adults and children requires a lot of computing power. Firstly identifying whether the source of the voice is an adult or a child, and then identifying the specific content, can reduce the computing power consumption of voice recognition.

[0021] Among them, speech category recognition is achieved by analyzing the acoustic features of speech (such as frequency, pitch, speaking speed, etc.). The purpose of this step is to distinguish between the main operator (usually an adult) and the non-operator (usually a child) to prevent children from triggering some functions by mistake. For example, adult voices have a lower pitch and a more stable speaking speed, while children's voices have a higher pitch and greater fluctuations. Through classification algorithms such as deep learning models or support vector machines, this judgment can be accurately completed and the speech category recognition results can be output. For example, sample speech data of adults and children is collected, and then the real source of the speech content is marked as adults or children. Supervised training is performed based on classification algorithms such as deep learning models or support vector machines to identify speech categories.

[0022] After determining the voice category, the specific content in the voice signal is analyzed, and the semantic information of the voice is extracted through natural language processing (NLP) technology to identify the core instructions of the voice. The recognition results include one of the different instruction types such as power on, power off, teaching, and reward. Teaching, for example, plays digital teaching content, and rewards, for example, plays reward videos such as cartoons. For example, the user's voice "start digital teaching" will be parsed as a teaching instruction, and "give the child a reward" will be parsed as a reward instruction. Through this classification, the operation target of the voice instruction is clarified.

[0023] Among them, according to the result of voice category recognition, the content of the voice command is recognized. For example, if it is recognized as an adult voice, an adult voice content recognition model trained based on a natural language processing algorithm is used for recognition. If it is recognized as a child voice, a child voice content recognition model trained based on a natural language processing algorithm is used for recognition. In this way, the voice recognition content can be transformed from an 8-category problem to a 2-category problem plus a 4-category problem, which can effectively reduce the size of the voice recognition model and the computing power consumed by voice recognition, making it lighter and faster. Training a voice content recognition model based on a natural language processing algorithm is a training method in the prior art and will not be described in detail here.

[0024] The voice category recognition result and the voice content recognition result are combined to obtain a voice command recognition result of the current voice command. For example, the voice command recognition result can be any one of adult power on, adult power off, adult teaching, adult reward, child power on, child power off, child teaching, and child reward.

[0025] Furthermore, when the voice command recognition results include adult voice and teaching, the voice command is determined to be a teaching command, the teaching content in the teaching command is displayed, and relevant teaching resources are retrieved according to the specific content in the command. For example, if the voice command is "learn 7", early education content with the number "7", such as videos, is played to conduct early education teaching.

[0026] By combining voice category recognition and voice content recognition, the robot's ability to respond to voice commands is significantly improved, making recognition more lightweight and reducing the computing power consumed by voice recognition.

[0027] In the embodiment of the present application, when the voice instruction recognition result includes adult voice and teaching, determining that the voice instruction is a teaching instruction, and displaying the teaching content in the teaching instruction to the user include: When the voice instruction recognition result includes adult voice and teaching, determining that the voice instruction is a teaching instruction, and identifying a teaching node in the voice instruction; The teaching content in the teaching node is displayed to the user.

[0028] In the embodiment of the present application, when the voice instruction input by the user is recognized as an adult voice and teaching, the instruction is determined to be a teaching instruction.

[0029] On this basis, the system recognizes and processes the teaching nodes in the voice command. For example, after recognizing the voice command "learn the number 7" as teaching and adult voice, the system recognizes and processes the teaching nodes in the voice command.

[0030] Teaching nodes refer to knowledge content units in teaching content, such as "number 7", "English letters A to D", etc. Teaching node identification in teaching instructions can be identified through natural language recognition algorithms, and the teaching content units therein are obtained as teaching nodes. Early childhood education courses can be pre-set in the early childhood education robot and multiple teaching nodes can be arranged for users to select and play through voice control.

[0031] After successfully parsing the teaching node, enter the teaching content display, call the teaching resources corresponding to the teaching node, and display them to the user. The teaching content can include various forms, such as picture display, animation demonstration and voice explanation. The specific display method is optimized according to the node type and the children's acceptance ability.

[0032] The embodiment of the present application utilizes voice category recognition to ensure accurate positioning of instructions, utilizes teaching node recognition to realize modular display of teaching content, and improves the intelligence level of early childhood education.

[0033] S200: After the teaching content is presented, receiving a user's feedback voice and collecting a user's feedback image; In the embodiment of the present application, after the teaching content is presented, the user's learning feedback information is obtained, for example, feedback information from children after reading the teaching content is collected.

[0034] The present application collects the user's feedback voice and action images through voice interaction and action interaction to identify the target of the synchronous interaction of voice and action, and then receives the user's feedback voice and collects the user's feedback image.

[0035] For example, after the teaching content is presented, the child can be prompted to provide voice feedback and action feedback, such as playing a prompt voice of "say the number you just saw and then use your hands to indicate it", and then collecting the voice feedback of the child and the video image of the child's action feedback. For example, the child can be prompted to stand in a specified position area and provide action feedback to improve the quality of feedback image collection.

[0036] S300: Perform content consistency check on the feedback voice and feedback image. After the consistency check passes, determine whether they are consistent with the teaching content, obtain early education processing results, and display them to the user, wherein the early education processing results include correct feedback or incorrect feedback.

[0037] like Figure 2 As shown, in the embodiment of the present application, after receiving the user's feedback voice and feedback image, the user feedback content in the two feedback information is compared through consistency check to ensure the semantic and behavioral consistency of the voice feedback and the image feedback. That is, it is determined whether the numbers expressed in the child's feedback voice and the numbers expressed in the action image are consistent, and then it is determined whether the child has mastered the teaching content.

[0038] After the consistency check is passed, the feedback content in the feedback voice and feedback image is compared to see if it is consistent with the displayed teaching content, the correctness of the user feedback is judged, and the early education processing result is obtained and displayed to the user. The early education processing result includes correct feedback or incorrect feedback. For example, if the teaching content is the number 7, and the content expressed by the user in the feedback voice and feedback image is 7, then the early education processing result with correct feedback is obtained. On the contrary, if the content expressed by the user in the feedback voice and feedback image is not 7, then the early education processing result with incorrect feedback is obtained.

[0039] The early education processing results are displayed to users, prompting children to complete the teaching and feedback of a complete teaching content.

[0040] Step S300 in the method provided in the embodiment of the present application includes: Converting the feedback speech into an image to obtain a speech image; Performing content consistency check on the voice image and the feedback image to obtain a consistency check result; When the consistency check result is inconsistent, display feedback error information; When the consistency check result is consistent, consistent feedback information is obtained, whether the consistent feedback information is consistent with the teaching content is determined, and the early childhood education processing result with correct feedback or incorrect feedback is obtained and displayed to the user.

[0041] In the embodiment of the present application, after receiving the user's feedback voice, the voice signal is processed by the voice-to-image conversion module and converted into a voice image that is easy to compare. In this way, the feedback voice is converted into image data of the voice image, and the feedback image is also image data. The consistency of the two image data is verified to improve the accuracy and efficiency of the verification.

[0042] In the embodiment of the present application, performing image conversion on the feedback speech to obtain a speech image includes: performing normalization processing on a speech signal sequence in the feedback speech to obtain a normalized speech signal sequence; Using the Gram angle field, performing polar coordinate conversion on the normalized speech signal sequence to obtain a polar coordinate sequence; According to the polar coordinate sequence, a GASF graph or a GADF graph is generated as a speech image.

[0043] In the embodiment of the present application, the Gram angular field algorithm is used to perform image conversion processing on the feedback speech.

[0044] In an embodiment of the present application, after receiving the user's feedback voice, the voice signal is first normalized. Normalization refers to adjusting the original voice signal to a standardized range through mathematical methods to eliminate the influence of volume fluctuations and background noise, making the signal more suitable for subsequent feature extraction and analysis. Specifically, the system scales the amplitude range of the voice signal and maps it to [-1,1] or other specified ranges, while filtering out environmental noise through a denoising algorithm to improve the clarity of the signal. This step generates a normalized voice signal sequence.

[0045] The Gramian Angular Field (GAF) algorithm is used to process the normalized speech signal sequence to complete the conversion from one-dimensional time series to two-dimensional image representation. Specifically, each data point in the normalized speech signal sequence is mapped to polar coordinates to generate a polar coordinate sequence. For example, the signal amplitude is mapped to the polar radius (i.e., the distance from the origin), and the time evolution of the signal is mapped to the polar angle (i.e., the angle relative to the polar axis). This representation method converts the time series into intuitive geometric features, making it more suitable for graphical representation and subsequent pattern analysis. Then, the Gramian Angular Field diagram is further generated, including two forms: GASF diagram (Gramian Angular Summation Field) and GADF diagram (Gramian Angular Difference Field). The GASF diagram represents the cumulative trend of the time series by calculating the cosine value of the polar angle; the GADF diagram reflects the rate of change of the time series by calculating the sine value of the polar angle. These two images capture the dynamic characteristics of the sequence from different dimensions, providing diverse inputs for subsequent pattern recognition.

[0046] Finally, the generated GASF graph or GADF graph is used as a two-dimensional image representation of the speech signal, which can intuitively reflect the amplitude and relative change trend of the speech signal over time. One of the GASF graph or GADF graph is selected as the speech image.

[0047] The speech image not only retains the time correlation and dynamic characteristics of the user's feedback speech signal, but also provides a uniform data type in the form of an image for subsequent consistency verification.

[0048] After the feedback voice image is converted into a voice image, a consistency check is performed on the voice image and the feedback image to determine whether the contents expressed by the user feedback in the voice image and the feedback image are consistent, thereby obtaining a consistency check result.

[0049] In the embodiment of the present application, performing content consistency check on the voice image and the feedback image to obtain a consistency check result includes: Collecting a set of sample speech images and a set of sample feedback images, marking the sample speech image feature vectors and the sample feedback image feature vectors according to the contents in the sample speech images and the sample feedback images, and obtaining a set of sample speech image feature vectors and a set of sample feedback image feature vectors; A twin network is used to construct a consistency verification model, wherein the consistency verification model includes a first feature recognition path and a second feature recognition path, and a comparator connecting the output layers of the first feature recognition path and the second feature recognition path; The sample speech image set, the sample feedback image set, the sample speech image feature vector set and the sample feedback image feature vector set are used to supervise the consistency verification model until convergence, wherein the comparator calculates the Euclidean distance between the speech image feature vector and the feedback image feature vector output by the output layer of the first feature recognition path and the second feature recognition path, and performs consistency discrimination supervised training; The speech image and the feedback image are input into the trained consistency verification model, and the recognition output is used to obtain the consistency verification result.

[0050] In the embodiment of the present application, a twin network model is used to verify the consistency of the content of the speech image and the feedback image.

[0051] Specifically, firstly, a consistency verification model for consistency verification of the speech image and the feedback image is trained, and the consistency verification model is trained by a twin neural network.

[0052] First, a sample speech image set and a sample feedback image set are collected. The sample speech image set is obtained by converting standard speech signals into images (such as GASF or GADF images), and the sample feedback image set is obtained by collecting and processing feedback actions or expression images during user interaction. Both are used to train the consistency verification model.

[0053] Furthermore, in the sample data preprocessing stage, the respective feature vectors are extracted and annotated according to the contents of the collected sample voice images and feedback images. Feature vector is a high-dimensional data representation form used to describe the characteristics of an image. In an embodiment of the present application, the feature vector reflects the specific content in the voice image and the feedback image, such as the number expressed by the user. For example, the feature vector is annotated according to the content of the number 7 in the voice image and the feedback image. In this way, a set of sample voice image feature vectors and a set of sample feedback image feature vectors are obtained by annotation.

[0054] Furthermore, a consistency verification model is constructed using a Siamese Network. A Siamese Network is a special neural network structure that includes two feature recognition paths, which are used to process voice images and feedback images respectively. The two feature recognition paths are trained using a convolutional neural network and an LSTM network respectively to extract and recognize spatial information and temporal information in the voice image and the feedback image, and output the corresponding sample voice image feature vector and sample feedback image feature vector. The consistency verification model also includes a comparator, which connects the output layer of the first feature recognition path and the second feature recognition path, and is used to calculate the Euclidean distance between the voice image feature vector and the feedback image feature vector output by the output layer of the first feature recognition path and the second feature recognition path, and then determines whether it is less than the Euclidean distance threshold, and then determines whether the contents in the voice image and the feedback image are consistent. If it is less than or equal to, then it is consistent, otherwise, then it is inconsistent.

[0055] The Euclidean distance threshold can be set according to the Euclidean distance between the sample speech image feature vector and the sample feedback image feature vector with the same content. For example, the average of the Euclidean distances between multiple groups of sample speech image feature vectors and sample feedback image feature vectors with the same content is set as the Euclidean distance threshold.

[0056] During the model training process, a supervised learning method is used to input the sample speech image set, the sample feedback image set, and the corresponding feature vector set into the twin network, identify the speech image feature vector and the feedback image feature vector, calculate the Euclidean distance between the speech image feature vector and the feedback image feature vector, determine whether it is less than the Euclidean distance threshold, generate the consistency judgment result, and compare it with the consistency label of the sample annotation (for example, consistent or inconsistent). The training goal is to minimize the error between the output and the labeled value, thereby optimizing the parameters of the twin network. After multiple iterations, the model finally converges, that is, the consistency judgment accuracy of its output reaches the expected standard, such as an accuracy rate of 95%.

[0057] After the consistency verification model training is completed, the currently converted speech image and feedback image are input into the trained consistency verification model, the feature vectors in the speech image and feedback image are identified and extracted, the Euclidean distance is calculated for judgment, and the consistency verification result is output.

[0058] Through the above content, based on the twin network training consistency verification model, the matching degree of voice feedback and image feedback can be accurately determined. This technology not only improves the processing ability of multimodal feedback, but also significantly enhances the robot's understanding of user intentions. Combined with the user's action interaction, it can improve the diversity of early childhood education recognition interaction and the effect of early childhood education.

[0059] In the embodiment of the present application, when the consistency check result is inconsistent, a feedback error message is displayed to the user. For example, when the user's voice answer does not match the number expressed by the gesture, a prompt "Please try again" may be displayed.

[0060] When the consistency check result is consistent, consistent feedback information is generated, that is, the content of user feedback in the voice image and the feedback image, such as consistent numbers, and the next teaching content comparison stage is entered to determine whether the consistent feedback information is consistent with the teaching content, obtain the processing results, and display them to the user.

[0061] In the embodiment of the present application, when the consistency check result is consistent, obtaining consistent feedback information, determining whether the consistent feedback information is consistent with the teaching content, obtaining the early education processing result, and displaying it to the user include: When the consistency check result is consistent, obtaining consistent feedback information in the voice image and the feedback image; Determining whether the consistent feedback information is consistent with the teaching content; If so, the correct early childhood education processing result is obtained, and if not, the wrong early childhood education processing result is obtained and displayed to the user.

[0062] In the embodiment of the present application, when the consistency check result is consistent, the consistent feedback information contained in the voice image and the feedback image is obtained. For example, a convolutional neural network is used to identify the feedback image, and the numbers contained therein are identified as consistent feedback information, or the content contained in the feature vector, such as the represented number, is extracted based on the feature vector of the voice image and the feature vector of the feedback image as consistent feedback information.

[0063] Compare the consistent feedback information with the displayed teaching content. Determine whether the content of the consistent feedback information is consistent with the teaching content, for example, whether the numbers in the consistent feedback information are the numbers taught in the teaching content, for example, whether they are all 7. If so, obtain the early education processing result with correct feedback, display it to the user, specifically display the correct feedback information, and prompt the user that the feedback is correct, such as by voice broadcasting "Correct answer!" or displaying a reward animation. If not, obtain the early education processing result with incorrect feedback, display it to the user, specifically display the feedback error information, and prompt the user that the feedback is wrong, such as a voice prompt "Please try again".

[0064] Through the above steps, the complete process from consistency verification of collected information of voice feedback and action feedback to feedback comparison and result display is realized. This method not only improves the early childhood education robot's ability to process user feedback, but also enhances the user's interactive experience through accurate correct and incorrect feedback.

[0065] In the embodiment of the present application, after receiving the voice command input by the user and performing recognition, the method further includes: When the voice command recognition result includes an adult voice or a child voice, and a power on or power off command, determining that the voice command is a power on or power off command, and powering on or powering off the device; When the voice instruction recognition result includes adult voice, and teaching or rewarding, determining that the voice instruction is a teaching instruction or a rewarding instruction, and performing teaching or rewarding; When the voice instruction recognition result includes children's voice, teaching or reward, the voice instruction is determined to be an invalid instruction, and a prompt message is generated for prompting.

[0066] In the embodiment of the present application, in order to ensure the active use of the early education robot and prevent children from using wrong instructions to affect the presentation of teaching or reward content, specific voice instructions are set.

[0067] Among them, when the voice command recognition result shows that the voice category is adult voice or child voice, and the content is to turn on or off the power, the command is determined to be a power on or off command, and the power is turned on or off according to the command. That is, both adults and children can control the power on and off of the early childhood education robot.

[0068] When the voice command recognition result includes adult voice, teaching or reward, the voice command is judged to be a teaching command or a reward command, and teaching or rewarding is performed, such as playing teaching content or rewarding content such as cartoons. That is, adults can control the teaching and rewarding content of the early childhood education robot.

[0069] When the voice command recognition results include children's voices, as well as teaching or rewards, the voice command is judged to be an invalid command and a prompt message is generated to prompt. Children are not allowed to control the teaching and reward content of the early education robot to prevent children from affecting the display of teaching or reward content and affecting the effect of early education. Prompt information such as voice prompts "Please let the parents operate" or "This function requires the help of parents".

[0070] Through the above steps, different voice categories and contents can be intelligently distinguished to ensure accurate triggering of functions and effective protection against misoperation.

[0071] In summary, the embodiments of the present application have at least the following technical effects: The embodiment of the present application proposes a method for voice processing of an early childhood education robot, which divides the input voice into adult voice and child voice through voice category recognition, and further combines voice content recognition, which significantly simplifies the voice processing process and improves the accuracy of command recognition. Compared with traditional voice recognition methods, the present invention can quickly identify teaching voices, avoid complex calculations of irrelevant voices, and reduce computing power consumption. The present application also uses the feature consistency of voice and children's action images for early childhood education and double verification, which effectively improves the diversity of interaction when using early childhood education robots, and improves children's early childhood education enthusiasm and early childhood education effects. The present application achieves the technical effect of improving the diversity and teaching effect of the interaction between early childhood education robots and children, and reducing the consumption of voice recognition computing power.

[0072] Embodiment 2, as Figure 3 As shown, based on the same inventive concept of a speech processing method for an early education robot provided in Embodiment 1, an embodiment of the present invention further provides an early education robot, comprising: The voice instruction parsing module 11 is used to receive the voice instruction input by the user, identify it, and when it is identified as a teaching instruction, display the teaching content in the teaching instruction to the user; A feedback information collection module 12, used for receiving user feedback voice and collecting user feedback images after the teaching content is presented; The feedback information processing module 13 is used to perform content consistency check on the feedback voice and feedback image. After the consistency check is passed, it is determined whether they are consistent with the teaching content, and the early education processing result is obtained and displayed to the user, wherein the early education processing result includes correct feedback or incorrect feedback.

[0073] In one embodiment, the voice instruction parsing module 11 is further used for: Receiving voice commands input by a user; Performing voice category recognition on the voice instruction to obtain a voice category recognition result, wherein the voice category recognition result includes one of an adult voice and a child voice; Performing content recognition on the voice command to obtain a voice content recognition result, wherein the voice content recognition result includes one of power on, power off, teaching and rewarding; Combining the voice category recognition result and the voice content recognition result to obtain a voice command recognition result; When the voice instruction recognition result includes adult voice and teaching, the voice instruction is determined to be a teaching instruction, and the teaching content in the teaching instruction is displayed to the user.

[0074] Wherein, when the voice instruction recognition result includes adult voice and teaching, determining that the voice instruction is a teaching instruction, and displaying the teaching content in the teaching instruction to the user, comprises: When the voice instruction recognition result includes adult voice and teaching, determining that the voice instruction is a teaching instruction, and identifying a teaching node in the voice instruction; The teaching content in the teaching node is displayed to the user.

[0075] In one embodiment, the feedback information processing module 13 is further used to: convert the feedback voice into an image to obtain a voice image; Performing content consistency check on the voice image and the feedback image to obtain a consistency check result; When the consistency check result is inconsistent, display feedback error information; When the consistency check result is consistent, consistent feedback information is obtained, whether the consistent feedback information is consistent with the teaching content is determined, and the early childhood education processing result with correct feedback or incorrect feedback is obtained and displayed to the user.

[0076] The step of converting the feedback voice into an image to obtain a voice image includes: Normalizing the speech signal sequence in the feedback speech to obtain a normalized speech signal sequence; Using the Gram angle field, performing polar coordinate conversion on the normalized speech signal sequence to obtain a polar coordinate sequence; According to the polar coordinate sequence, a GASF graph or a GADF graph is generated as a speech image.

[0077] The process of performing content consistency check on the voice image and the feedback image to obtain a consistency check result includes: Collecting a set of sample speech images and a set of sample feedback images, marking the sample speech image feature vectors and the sample feedback image feature vectors according to the contents in the sample speech images and the sample feedback images, and obtaining a set of sample speech image feature vectors and a set of sample feedback image feature vectors; A twin network is used to construct a consistency verification model, wherein the consistency verification model includes a first feature recognition path and a second feature recognition path, and a comparator connecting the output layers of the first feature recognition path and the second feature recognition path; The sample speech image set, the sample feedback image set, the sample speech image feature vector set and the sample feedback image feature vector set are used to supervise the consistency verification model until convergence, wherein the comparator calculates the Euclidean distance between the speech image feature vector and the feedback image feature vector output by the output layer of the first feature recognition path and the second feature recognition path, and performs consistency discrimination supervised training; The speech image and the feedback image are input into the trained consistency verification model, and the recognition output is used to obtain the consistency verification result.

[0078] Wherein, when the consistency check result is consistent, obtaining consistent feedback information, judging whether the consistent feedback information is consistent with the teaching content, obtaining the early education processing result, and displaying it to the user, including: When the consistency check result is consistent, obtaining consistent feedback information in the voice image and the feedback image; Determining whether the consistent feedback information is consistent with the teaching content; If so, the correct early childhood education processing result is obtained, and if not, the wrong early childhood education processing result is obtained and displayed to the user.

[0079] Wherein, after receiving the voice command input by the user and performing recognition, the method further includes: When the voice command recognition result includes an adult voice or a child voice, and a power on or power off command, determining that the voice command is a power on or power off command, and powering on or powering off the device; When the voice instruction recognition result includes adult voice, and teaching or rewarding, determining that the voice instruction is a teaching instruction or a rewarding instruction, and performing teaching or rewarding; When the voice instruction recognition result includes children's voice, teaching or reward, the voice instruction is determined to be an invalid instruction, and a prompt message is generated for prompting.

[0080] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0081] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0082] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A speech processing method for an early childhood education robot, characterized in that: The method comprises: Receiving a voice command input by a user for recognition, and when the voice command is recognized as a teaching command, displaying the teaching content in the teaching command to the user; After the teaching content is presented, receiving user feedback voice and collecting user feedback images; The feedback voice and feedback image are checked for content consistency. After the consistency check is passed, it is determined whether they are consistent with the teaching content, and the early education processing result is obtained and displayed to the user, wherein the early education processing result includes correct feedback or incorrect feedback.

2. The speech processing method of the early childhood education robot according to claim 1, characterized in that: Receiving a voice command input by a user for recognition, and when the voice command is recognized as a teaching voice command, displaying the teaching content in the teaching command to the user, including: Receiving voice commands input by a user; Performing voice category recognition on the voice instruction to obtain a voice category recognition result, wherein the voice category recognition result includes one of an adult voice and a child voice; Performing content recognition on the voice command to obtain a voice content recognition result, wherein the voice content recognition result includes one of power on, power off, teaching and rewarding; Combining the voice category recognition result and the voice content recognition result to obtain a voice command recognition result; When the voice instruction recognition result includes adult voice and teaching, the voice instruction is determined to be a teaching instruction, and the teaching content in the teaching instruction is displayed to the user.

3. The speech processing method of the early childhood education robot according to claim 2, characterized in that: When the voice instruction recognition result includes adult voice and teaching, determining that the voice instruction is a teaching instruction, and displaying the teaching content in the teaching instruction to the user, includes: When the voice instruction recognition result includes adult voice and teaching, determining that the voice instruction is a teaching instruction, and identifying a teaching node in the voice instruction; The teaching content in the teaching node is displayed to the user.

4. The speech processing method of the early childhood education robot according to claim 1, characterized in that: Performing content consistency check on the feedback voice and feedback image, and after the consistency check passes, determining whether they are consistent with the teaching content, obtaining the early education processing result, and displaying it to the user, including: Converting the feedback speech into an image to obtain a speech image; Performing content consistency check on the voice image and the feedback image to obtain a consistency check result; When the consistency check result is inconsistent, display feedback error information; When the consistency check result is consistent, consistent feedback information is obtained, whether the consistent feedback information is consistent with the teaching content is determined, and the early childhood education processing result with correct feedback or incorrect feedback is obtained and displayed to the user.

5. The speech processing method of the early childhood education robot according to claim 4, characterized in that: Converting the feedback voice into an image to obtain a voice image includes: Normalizing the speech signal sequence in the feedback speech to obtain a normalized speech signal sequence; Using the Gram angle field, performing polar coordinate conversion on the normalized speech signal sequence to obtain a polar coordinate sequence; According to the polar coordinate sequence, a GASF graph or a GADF graph is generated as a speech image.

6. The speech processing method of the early childhood education robot according to claim 4, characterized in that: Performing content consistency check on the voice image and the feedback image to obtain a consistency check result includes: Collecting a set of sample speech images and a set of sample feedback images, marking the sample speech image feature vectors and the sample feedback image feature vectors according to the contents in the sample speech images and the sample feedback images, and obtaining a set of sample speech image feature vectors and a set of sample feedback image feature vectors; A twin network is used to construct a consistency verification model, wherein the consistency verification model includes a first feature recognition path and a second feature recognition path, and a comparator connecting the output layers of the first feature recognition path and the second feature recognition path; The sample speech image set, the sample feedback image set, the sample speech image feature vector set and the sample feedback image feature vector set are used to supervise the consistency verification model until convergence, wherein the comparator calculates the Euclidean distance between the speech image feature vector and the feedback image feature vector output by the output layer of the first feature recognition path and the second feature recognition path, and performs consistency discrimination supervised training; The speech image and the feedback image are input into the trained consistency verification model, and the recognition output is used to obtain the consistency verification result.

7. The speech processing method of the early childhood education robot according to claim 4, characterized in that: When the consistency check result is consistent, obtaining consistent feedback information, determining whether the consistent feedback information is consistent with the teaching content, obtaining the early education processing result, and displaying it to the user, including: When the consistency check result is consistent, obtaining consistent feedback information in the voice image and the feedback image; Determining whether the consistent feedback information is consistent with the teaching content; If so, the correct early childhood education processing result is obtained, and if not, the wrong early childhood education processing result is obtained and displayed to the user.

8. The speech processing method of the early childhood education robot according to claim 2, characterized in that: After receiving a voice command input by a user and performing recognition, the method further includes: When the voice command recognition result includes an adult voice or a child voice, and a power on or power off command, determining that the voice command is a power on or power off command, and powering on or powering off the device; When the voice instruction recognition result includes adult voice, and teaching or rewarding, determining that the voice instruction is a teaching instruction or a rewarding instruction, and performing teaching or rewarding; When the voice instruction recognition result includes children's voice, teaching or reward, the voice instruction is determined to be an invalid instruction, and a prompt message is generated for prompting.

9. An early childhood education robot, characterized in that: The early childhood education robot comprises: A voice instruction parsing module is used to receive and recognize the voice instruction input by the user, and when it is recognized as a teaching instruction, display the teaching content in the teaching instruction to the user; A feedback information collection module, used to receive user feedback voice and collect user feedback images after the teaching content is presented; The feedback information processing module is used to perform content consistency verification on the feedback voice and feedback image. After the consistency verification is passed, it is determined whether they are consistent with the teaching content, and the early education processing result is obtained and displayed to the user, wherein the early education processing result includes correct feedback or incorrect feedback.

10. The early childhood education robot according to claim 9, characterized in that: The voice command parsing module is also used for: Receiving voice commands input by a user; Performing voice category recognition on the voice instruction to obtain a voice category recognition result, wherein the voice category recognition result includes one of an adult voice and a child voice; Performing content recognition on the voice command to obtain a voice content recognition result, wherein the voice content recognition result includes one of power on, power off, teaching and rewarding; Combining the voice category recognition result and the voice content recognition result to obtain a voice command recognition result; When the voice instruction recognition result includes adult voice and teaching, the voice instruction is determined to be a teaching instruction, and the teaching content in the teaching instruction is displayed.

Images