Interactive infant education system and method based on motion capture

Abstract

The invention relates to the technical field of infant learning and education, in particular to an interactive infant education system and method based on action capture, and the system comprises the steps: accessing infant feedback action information and interactive robot education instruction information at a plurality of preset moments in a preset period from an infant education cloud platform server; respectively executing type corresponding label distribution on the infant feedback action information and the interactive robot education instruction information at the plurality of preset moments to obtain an infant feedback information type corresponding label sequence and an interactive robot education behavior type corresponding label sequence; executing type feedback interaction feature acquisition on the label sequence corresponding to the infant feedback information type and the label sequence corresponding to the interactive robot educational behavior type so as to acquire an infant mental adaptation reliability matrix; and through the infant mental adaptation configuration reliability matrix, determining an evaluation record of infant mental adaptation development. The method provides targeted learning content for the user, pays attention to non-intellectual factors, and pays attention to whether adaptive education for children is reasonable or not.

Description

Technical Field

[0001] The present application relates to the technical field of early childhood learning and education, and in particular to an interactive early childhood education system and method based on motion capture. Background Art

[0002] The role of early childhood education evaluation is mainly reflected in the following aspects: First, it can provide a basis for early childhood education decision-making, help early childhood education managers understand the effectiveness and quality of early childhood education activities, and make more reasonable decisions. Secondly, early childhood education evaluation can promote the professional development of early childhood education robots. Through the evaluation results, early childhood education robots can understand their own teaching effects and then adjust teaching methods and content. Thirdly, early childhood education evaluation can serve as feedback on learning outcomes for young children, helping them recognize their strengths and areas for improvement. Finally, early childhood education evaluation can also enhance the transparency of early childhood education and the public's trust in the quality of early childhood education.

[0003] Most of the current children's robots simply store course content in the cloud, and use data processing to allow the robot to play the course content through voice when needed. At the same time, many children's robots focus on entertainment activities for children, play simple content, cannot provide users with targeted learning content, ignore non-intellectual factors, and traditional evaluation methods often only focus on the knowledge mastery of children, but not enough on whether the adaptive education for children is reasonable. Summary of the invention

[0004] To achieve the above objectives, this application provides the following technical solutions:

[0005] According to a first aspect of the present invention, the present invention claims protection for an interactive early childhood education system based on motion capture, comprising:

[0006] A preschool education cloud platform data access unit, used to access preschool feedback action information and interactive robot education instruction information at multiple preset moments within a preset period from a preschool education cloud platform server, wherein the preschool feedback action information includes the preschool's sight, facial expression, upper limb action, lower limb action, voice characteristics and interactive feedback, and the interactive robot education instruction information includes voice education instructions, action education instructions and imitation education instructions;

[0007] A continuous action information corresponding label allocation unit is used to respectively perform type corresponding label allocation on the child feedback action information and the interactive robot education instruction information at the plurality of preset moments to obtain a child feedback information type corresponding label sequence and an interactive robot education behavior type corresponding label sequence;

[0008] A type feedback interaction unit, used for performing type feedback interaction feature acquisition on the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot educational behavior type to obtain a child mental adaptation confidence matrix;

[0009] A mental development evaluation unit, used to determine the evaluation record of the child's mental adaptation development through the child's mental adaptation configuration reliability matrix;

[0010] Wherein, the mental development assessment unit includes:

[0011] A correction unit, used for performing spatiotemporal dependency restriction of data propagation breadth on the child's mental fitness configuration confidence matrix to obtain a corrected child's mental fitness configuration confidence matrix;

[0012] The evaluation record output unit is used to obtain the evaluation record by configuring the mental adaptation reliability matrix of the child according to the mental development evaluator through the classifier, and the evaluation record is used to indicate whether to transmit the mental development reminder to the guardian.

[0013] Furthermore, the continuous action information corresponds to a label allocation unit, including:

[0014] A child feedback information type evaluation unit, configured to arrange the child feedback action information at the plurality of preset moments into a child feedback information input tuple according to time and scene aspects, and then obtain a child feedback information type corresponding label sequence according to a child feedback information type corresponding label distributor using a deep learning model;

[0015] An interactive robot educational behavior type evaluation unit is used to arrange the interactive robot education instruction information of multiple preset moments into interactive robot education behavior input tuples according to time and scene aspects, and then obtain the interactive robot education behavior type corresponding label sequence based on the education behavior type corresponding label distributor through a deep learning model.

[0016] Furthermore, the type feedback interaction unit includes:

[0017] A behavior sequence standardization processing unit, used for performing behavior sequence association adjustment on the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot educational behavior type to obtain a label sequence corresponding to the child feedback information type after association and a label sequence corresponding to the interactive robot educational behavior type after association;

[0018] The feedback prediction unit is used to calculate the feedback prediction of the label sequence corresponding to the child's feedback information type after the association relative to the label sequence corresponding to the interactive robot educational behavior type after the association to obtain the child's mental adaptation confidence matrix.

[0019] Furthermore, the behavior sequence standardization processing unit is used to:

[0020] The label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot educational behavior type are embedded in the co-space associator through the fully connected layer to obtain the label sequence corresponding to the child feedback information type after association and the label sequence corresponding to the interactive robot educational behavior type after association.

[0021] Furthermore, the feedback prediction unit is used to:

[0022] The label sequence corresponding to the type of child feedback information after association and the inverse sequence of the label sequence corresponding to the type of interactive robot educational behavior after association are multiplied to obtain the child mental adaptation configuration confidence matrix.

[0023] Furthermore, the correction unit is used to:

[0024] Calculating the variance of a node group consisting of node values ​​at all positions of the child mental fitness configuration confidence matrix to obtain the node value variance of the child mental fitness configuration confidence matrix;

[0025] Calculating the square of the absolute value of the node value at each position of the child mental fitness configuration confidence matrix and then performing a likelihood operation to obtain a likelihood matching child mental fitness configuration confidence matrix;

[0026] Performing cumulative summation on all node values ​​of the likelihood matching child mental fitness configuration confidence matrix to obtain a sum value of the likelihood matching child mental fitness configuration confidence matrix;

[0027] The sum of the likelihood matching child mental fitness configuration reliability matrix is ​​divided by twice the node value variance of the child mental fitness configuration reliability matrix and then multiplied by the node value of each position of the child mental fitness configuration reliability matrix to obtain the modified child mental fitness configuration reliability matrix.

[0028] According to a second aspect of the present invention, the present invention claims protection for an interactive early childhood education method based on motion capture, comprising:

[0029] Accessing the child feedback action information and interactive robot education instruction information at multiple preset moments within a preset period from the early childhood education cloud platform server, wherein the child feedback action information includes the child's sight, facial expression, upper limb action, lower limb action, voice characteristics and interactive feedback, and the interactive robot education instruction information includes voice education instruction, action education instruction and imitation education instruction;

[0030] Perform type-corresponding label assignment on the child feedback action information and the interactive robot educational instruction information at the plurality of preset moments to obtain a child feedback information type-corresponding label sequence and an interactive robot educational behavior type-corresponding label sequence;

[0031] Performing type feedback interaction feature acquisition on the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot educational behavior type to obtain the child mental adaptation confidence matrix;

[0032] Determine the evaluation record of the child's mental adaptation development through the child's mental adaptation reliability matrix;

[0033] The evaluation record of the development of children's mental adaptation is determined by the children's mental adaptation reliability matrix, including:

[0034] Performing spatiotemporal dependency restriction of data propagation breadth on the child's mental fitness configuration confidence matrix to obtain a modified child's mental fitness configuration confidence matrix;

[0035] The child's mental adaptation reliability matrix is ​​applied to the mental development evaluator through the classifier to obtain the evaluation record, and the evaluation record is used to indicate whether to transmit a mental development reminder to the guardian.

[0036] Further, type-corresponding label assignment is performed on the child feedback action information and the interactive robot education instruction information at the plurality of preset moments to obtain a child feedback information type-corresponding label sequence and an interactive robot education behavior type-corresponding label sequence, including:

[0037] Arrange the child feedback action information at the plurality of preset moments according to time and scene aspects into a child feedback information input tuple, and then obtain a label sequence corresponding to the child feedback information type according to a child feedback information type corresponding label distributor using a deep learning model;

[0038] After arranging the interactive robot education instruction information at multiple preset moments into interactive robot education behavior input tuples according to time and scene aspects, a label distributor corresponding to the education behavior type through a deep learning model is used to obtain a label sequence corresponding to the interactive robot education behavior type.

[0039] The application relates to an interactive early childhood education system and method based on motion capture, which accesses the early childhood feedback action information and interactive robot education instruction information of multiple preset moments within a preset period from the early childhood education cloud platform server, respectively performs type-corresponding label allocation on the early childhood feedback action information and interactive robot education instruction information of multiple preset moments to obtain the early childhood feedback information type corresponding label sequence and the interactive robot education behavior type corresponding label sequence; performs type feedback interaction feature acquisition on the early childhood feedback information type corresponding label sequence and the interactive robot education behavior type corresponding label sequence to obtain the early childhood mental adaptation configuration reliability matrix; through the early childhood mental adaptation configuration reliability matrix, determines the evaluation record of the early childhood mental adaptation development. The present invention provides users with targeted learning content, pays attention to non-intellectual factors, and attaches importance to whether the adaptive education of early childhood is reasonable. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A structural block diagram of an interactive early childhood education system based on motion capture claimed in an embodiment of the present application;

[0041] Figure 2 A second structural block diagram of an interactive early childhood education system based on motion capture claimed in an embodiment of the present application;

[0042] Figure 3 A third structural block diagram of an interactive early childhood education system based on motion capture claimed in an embodiment of the present application;

[0043] Figure 4 A fourth structural block diagram of an interactive early childhood education system based on motion capture claimed in an embodiment of the present application;

[0044] Figure 5 This is a workflow diagram of an interactive early childhood education method based on motion capture as claimed in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field without creative work through the embodiments in the present application are within the scope of protection of this application.

[0046] Figure 1 FIG. 1 is a block diagram of an interactive early childhood education system based on motion capture according to an embodiment of the present application. Figure 1As shown, according to the embodiment of the present application, the interactive early childhood education system based on motion capture includes: an early childhood education cloud platform data access unit 110, which is used to access the early childhood education cloud platform server at multiple preset moments in a preset period. The early childhood feedback action information includes the child's line of sight, facial expression, upper limb action, lower limb action, sound characteristics and interactive feedback, and the interactive robot education instruction information includes voice education instructions, action education instructions and imitation education instructions; a continuous action information corresponding label allocation unit 120, which is used to perform type corresponding label allocation on the child feedback action information and the interactive robot education instruction information at the multiple preset moments to obtain a child feedback information type corresponding label sequence and an interactive robot education behavior type corresponding label sequence; a type feedback interaction unit 130, which is used to perform type feedback interaction feature acquisition on the child feedback information type corresponding label sequence and the interactive robot education behavior type corresponding label sequence to obtain a child mental adaptation configuration confidence matrix; a mental development evaluation unit 140, which is used to determine the evaluation record of the child's mental adaptation development through the child mental adaptation configuration confidence matrix.

[0047] Since the early childhood education cloud platform server needs to interact with various external devices, there is a reasonable risk that malicious attackers will exploit vulnerabilities to remotely control or tamper with children's information, which may cause malfunctions or accidents to children. In response to the above technical problems, the technical concept of this application is to use artificial intelligence technology through deep learning to perform data evaluation on the educational instruction information of the interactive robot and the feedback action information of the children, obtain the educational behavior characteristics of the interactive robot and the feedback information characteristics of the children, and characterize the reliability of the children's mental adaptation through the type feedback interaction characteristics between the two, so as to intelligently judge whether the children have reasonable mental development and provide corresponding development reminders. In this way, guardians can be helped to discover potential mental risks in a timely manner, so as to take corresponding measures in a timely manner to ensure the mental nature of the children's feedback.

[0048] In the above-mentioned interactive early childhood education system based on motion capture, the early childhood education cloud platform data access unit 110 is used to access the early childhood education cloud platform server for multiple preset moments of the early childhood education feedback action information and interactive robot education instruction information within a preset period, wherein the early childhood education feedback action information includes the child's sight, facial expression, upper limb action, lower limb action, sound characteristics and interactive feedback, and the interactive robot education instruction information includes voice education instruction, action education instruction and imitation education instruction. It should be understood that the data such as the child's sight, facial expression and upper limb action directly reflect the child's power performance and feedback information; while the voice education instruction, action education instruction and imitation education instruction of the interactive robot reflect the operation intention of the interactive robot, and the child feedback action information and the interactive robot education instruction information together constitute the complete information of the child's feedback and mental adaptation. In the technical solution of the present application, by obtaining the child feedback action information and interactive robot education instruction information at multiple preset moments within the preset period, the actual operation status of the child at different times and the operation behavior of the interactive robot can be fully reflected. Furthermore, artificial intelligence technology can be used to evaluate the dynamic characteristics of children and educational behaviors over time, capture the dynamic correspondence between children and educational behaviors, identify abnormal or inconsistent behavior patterns, and thus determine whether children have reasonable potential for mental development. This evaluation method based on type data can effectively provide a reliable data basis for subsequent mental development evaluation.

[0049] In the above-mentioned interactive early childhood education system based on motion capture, the continuous action information corresponding label allocation unit 120 is used to perform type corresponding label allocation on the early childhood feedback action information and interactive robot education instruction information at the plurality of preset moments to obtain a type corresponding label sequence of early childhood feedback information and a type corresponding label sequence of interactive robot education behavior. Figure 2 FIG. 1 is a block diagram of a label assignment unit for continuous motion information in an interactive early childhood education system based on motion capture according to an embodiment of the present application. Figure 2 As shown, the continuous action information corresponding label allocation unit 120 includes: a child feedback information type evaluation unit 121, which is used to arrange the child feedback action information of the multiple preset moments into child feedback information input tuples according to time and scene aspects, and then obtain the child feedback information type corresponding label sequence according to the child feedback information type corresponding label distributor through the deep learning model; an interactive robot educational behavior type evaluation unit 122, which is used to arrange the interactive robot educational instruction information of the multiple preset moments into interactive robot educational behavior input tuples according to time and scene aspects, and then obtain the interactive robot educational behavior type corresponding label sequence according to the educational behavior type corresponding label distributor through the deep learning model.

[0050] Specifically, the child feedback information type evaluation unit 121 is used to arrange the child feedback action information of the plurality of preset moments into a child feedback information input tuple according to the time aspect and the scene aspect, and then obtain the child feedback information type corresponding label sequence according to the child feedback information type corresponding label distributor through the deep learning model. It should be understood that considering that the child feedback action information contains multiple parameters (such as sight, facial expression, upper limb movement, etc.), these parameters present different numerical changes at different times. In other words, each parameter in the child feedback action information has a reasonable type change corresponding relationship in terms of time. Through this, in the technical solution of the present application, in order to effectively obtain the type corresponding label in the child feedback action information, the type evaluation technology of the deep learning model is used to perform evaluation processing on the child feedback action information. Specifically, first, the child feedback action information of the plurality of preset moments is arranged according to the time aspect and the scene aspect to form a two-dimensional child feedback information input tuple. In this way, each row of the child feedback information input tuple corresponds to a moment, and each column corresponds to a parameter, thereby integrating the type distribution information of the child feedback action information and retaining the corresponding relationship between the various parameters. Then, the child feedback information input tuple is input into the child feedback information type corresponding label distributor through the deep learning model. The child feedback information type corresponding label distributor uses the powerful feature acquisition capability of deep learning to perform local feature learning and spatial relationship acquisition on the child feedback information input tuple based on convolution operations, pooling operations and other techniques, so as to capture the type dependency relationship between various parameters in the child feedback action information, and mine the dynamic characteristics of the child feedback information changing over time, fully reflecting the inherent laws and patterns of the child feedback information, and providing effective data support for subsequent reliability evaluation and development evaluation.

[0051] Specifically, the interactive robot educational behavior type evaluation unit 122 is used to arrange the interactive robot educational instruction information at multiple preset moments into interactive robot educational behavior input tuples according to the time aspect and the scene aspect, and then obtain the interactive robot educational behavior type corresponding label sequence according to the educational behavior type corresponding label distributor through the deep learning model. It should be understood that the educational instruction information of the interactive robot has obvious type characteristics, that is, the operating behavior of the interactive robot changes continuously over time. Therefore, similarly, the interactive robot educational instruction information at multiple preset moments is arranged in the form of tuples according to the time aspect and the scene aspect to maintain the continuity and dependence of each interactive robot educational behavior parameter in time. Then use deep learning to perform type-dependent feature mining on it to comprehensively consider the mutual influence and type correspondence between these parameters, so as to obtain more comprehensive educational behavior characteristics, so as to fully understand the dynamic characteristics of the interactive robot educational behavior over time.

[0052] In the above-mentioned interactive early childhood education system based on motion capture, the type feedback interaction unit 130 is used to perform type feedback interaction feature acquisition on the label sequence corresponding to the type of the early childhood feedback information and the label sequence corresponding to the type of the interactive robot educational behavior to obtain the confidence matrix of the early childhood mental adaptation. It should be understood that the label sequence corresponding to the type of the early childhood feedback information reflects the performance and operation information of the early childhood itself, while the label sequence corresponding to the type of the interactive robot educational behavior reveals the operating characteristics and behavior patterns of the interactive robot in the educational process. In order to comprehensively evaluate the reliability of the early childhood mental adaptation, it is necessary to comprehensively consider the influence of the operating behavior of the interactive robot on the early childhood feedback information. Therefore, in the technical solution of the present application, the type interaction evaluation is further performed on the label sequence corresponding to the type of the early childhood feedback information and the label sequence corresponding to the type of the interactive robot educational behavior to capture the type feedback interaction characteristics between the two.

[0053] Figure 3 FIG. 1 is a block diagram of a type feedback interaction unit in an interactive early childhood education system based on motion capture according to an embodiment of the present application. Figure 3 As shown, the type feedback interaction unit 130 includes: a behavior sequence standardization processing unit 131, which is used to perform behavior sequence association adjustment on the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot educational behavior type to obtain the label sequence corresponding to the child feedback information type after association and the label sequence corresponding to the interactive robot educational behavior type after association; a feedback prediction unit 132, which is used to calculate the feedback prediction of the label sequence corresponding to the child feedback information type after association relative to the label sequence corresponding to the interactive robot educational behavior type after association to obtain the child mental adaptation configuration confidence matrix.

[0054] Specifically, the behavior sequence standardization processing unit 131 is used to perform behavior sequence association adjustment on the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot education behavior type to obtain the label sequence corresponding to the child feedback information type after association and the label sequence corresponding to the interactive robot education behavior type after association. It should be understood that considering that the child feedback action information and the interactive robot education instruction information come from different sensors and data sources, they have different data formats and features, resulting in reasonable differences between the two in the behavior sequence, and it is impossible to directly perform feedback interaction evaluation. Therefore, it is necessary to further perform association adjustment on the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot education behavior type to eliminate the difference in features, so that the two can perform effective interaction evaluation in the same behavior sequence. In a specific example of the present application, the processing method of performing behavior sequence association adjustment on the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot education behavior type is to embed the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot education behavior type according to the co-space embedding associator through the fully connected layer to obtain the label sequence corresponding to the child feedback information type after association and the label sequence corresponding to the interactive robot education behavior type after association. Specifically, the fully connected layer performs linear matching on the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot educational behavior type to associate the two into the same shared space and eliminate differences in features, so that the two can perform interactive evaluation in the same behavior sequence, laying the foundation for further type interactive feedback evaluation.

[0055] Specifically, the feedback prediction unit 132 is used to calculate the feedback prediction of the label sequence corresponding to the type of the child's feedback information after the association relative to the label sequence corresponding to the type of the interactive robot's educational behavior after the association to obtain the child's mental adaptation configuration confidence matrix. It should be understood that in the child's mental adaptation system, there is a reasonable and close type interaction relationship between the interactive robot's educational behavior and the child's feedback information. According to the calculation of the feedback prediction of the label sequence corresponding to the type of the child's feedback information after the association relative to the label sequence corresponding to the type of the interactive robot's educational behavior after the association, the influence of the interactive robot's educational behavior on the child's feedback information and the feedback of the child's feedback information on the interactive robot's educational behavior can be further revealed, thereby reflecting the child's mental adaptation performance under different educational behaviors. For example, if the child's vision suddenly becomes unfocused and the action education instruction does not change accordingly, it may mean that the child has a reasonable development of being adapted to the unnatural mind. According to this method, the child's mental adaptation configuration confidence matrix can fully tap the interaction relationship between the child and the interactive robot, providing an important basis for the subsequent evaluation of the child's mental adaptation and mental development. In a specific example of the present application, the feedback prediction unit 132 is used to: multiply the label sequence corresponding to the child feedback information type after association and the inverse sequence of the label sequence corresponding to the interactive robot educational behavior type after association to obtain the child mental adaptation configuration confidence matrix.

[0056] In the above-mentioned interactive early childhood education system based on motion capture, the mental development evaluation unit 140 is used to determine the evaluation record of the early childhood mental adaptation development through the early childhood mental adaptation reliability matrix. Figure 4 FIG. 1 is a block diagram of an intellectual development evaluation unit in an interactive early childhood education system based on motion capture according to an embodiment of the present application. Figure 4 As shown, the mental development evaluation unit 140 includes: a correction unit 141, which is used to perform spatiotemporal dependency restrictions on the child's mental fitness configuration reliability matrix through data propagation breadth to obtain a corrected child's mental fitness configuration reliability matrix; an evaluation record output unit 142, which is used to obtain the evaluation record based on the child's mental fitness configuration reliability matrix according to the mental development evaluator through the classifier, and the evaluation record is used to indicate whether to transmit a mental development reminder to the guardian.

[0057] Specifically, the correction unit 141 is used to perform spatiotemporal dependency restrictions on the child's mental adaptation configuration reliability matrix through the data propagation breadth to obtain the modified child's mental adaptation configuration reliability matrix. The child's mental adaptation configuration reliability matrix is ​​subjected to spatiotemporal dependency restrictions through the data propagation breadth to obtain the modified child's mental adaptation configuration reliability matrix. In particular, in the technical solution of the present application, it is considered that the evaluation of the reliability of the child's mental adaptation needs to comprehensively consider the child's feedback action information and the interactive robot education instruction information. Therefore, if the data propagation breadth of the child's mental adaptation configuration reliability matrix can be improved, data from different sources can be better integrated, so that the model can fully understand the child's mental adaptation situation, thereby improving the effectiveness of mental development evaluation. Specifically, the child's mental adaptation configuration reliability matrix captures the type correspondence between data, including the correspondence between child feedback information and interactive robot education behavior. According to improving the data propagation breadth of the child's mental adaptation configuration reliability matrix, type corresponding labels can be better obtained, so that the model can more effectively evaluate the reliability of the child's mental adaptation. At the same time, evaluating the reliability of the mental adaptation of young children requires considering the feedback relationship between the information of young children and the behavior of the interactive robot. Improving the data dissemination breadth of the mental adaptation confidence matrix of the young children can help the model predict feedback more effectively, thereby more effectively evaluating the mental development of young children. The final mental development evaluation needs to be based on the characteristic information of the mental adaptation confidence matrix of the young children to effectively judge whether there is reasonable mental development and transmit reminders. Therefore, improving the data dissemination breadth of the mental adaptation confidence matrix of the young children can improve the effectiveness and reliability of the evaluation model, thereby more effectively protecting the minds of young children and passengers. Through this, the mental adaptation confidence matrix of the young children is further restricted by the spatiotemporal dependence of the data dissemination breadth.

[0058] Specifically, the correction unit 141 is used to: calculate the variance of the node group composed of the node values ​​of all positions of the child mental fitness configuration reliability matrix to obtain the node value variance of the child mental fitness configuration reliability matrix; calculate the square of the absolute value of the node value of each position of the child mental fitness configuration reliability matrix and then perform a likelihood operation to obtain a likelihood matching child mental fitness configuration reliability matrix; perform cumulative summation on all node values ​​of the likelihood matching child mental fitness configuration reliability matrix to obtain the sum value of the likelihood matching child mental fitness configuration reliability matrix; divide the sum value of the likelihood matching child mental fitness configuration reliability matrix by twice the node value variance of the child mental fitness configuration reliability matrix and then multiply it by the node value of each position of the child mental fitness configuration reliability matrix to obtain the corrected child mental fitness configuration reliability matrix.

[0059] That is to say, the child mental fitness configuration confidence matrix is ​​subjected to the spatiotemporal dependency restriction of the data propagation breadth to obtain the modified child mental fitness configuration confidence matrix, including: the child mental fitness configuration confidence matrix is ​​subjected to the spatiotemporal dependency restriction of the data propagation breadth to obtain the modified child mental fitness configuration confidence matrix according to the following correction formula, wherein the correction formula is:

[0060] ;

[0061] in, Explain the reliability matrix of the children's mental fitness configuration, This is the first The node value of the position, Description Node Value Set The variance of , They are the confidence matrices of the children's mental adaptability The width and height of Explain the base 2 likelihood, This is the first The node value of the position.

[0062] Here, in order to improve the data propagation breadth of the child mental fitness configuration confidence matrix, in the technical solution of the present application, the child mental fitness configuration confidence matrix is ​​subjected to spatiotemporal dependency restrictions through the data propagation breadth, and the likelihood value of the absolute phase stacking of the node values ​​at each position in the child mental fitness configuration confidence matrix is ​​used to illustrate the structural information of the child mental fitness configuration confidence matrix at the position, and then the structural information of the node values ​​at all positions in the child mental fitness configuration confidence matrix is ​​aggregated according to a wavelet-like function, and the child mental fitness configuration confidence matrix is ​​subjected to stacking processing through the feature distribution space by using the set variance of the structural information and the feature distribution of the child mental fitness configuration confidence matrix. In this way, it helps to maintain the spatial relationship between the features in the child mental fitness configuration confidence matrix, thereby performing spatiotemporal dependency restrictions on the child mental fitness configuration confidence matrix to improve the data propagation breadth of the child mental fitness configuration confidence matrix.

[0063] Specifically, the evaluation record output unit 142 is used to obtain the evaluation record according to the mental development evaluator through the classifier according to the mental fitness configuration reliability matrix of the child, and the evaluation record is used to indicate whether to transmit the mental development reminder to the guardian. It should be understood that the mental development evaluator through the classifier can effectively identify the mental development information in the input data after being trained with a large amount of data. Specifically, after receiving the mental fitness configuration reliability matrix of the child, the mental development evaluator through the classifier performs classification judgment on the type feedback correspondence pattern between the child feedback information contained in the mental fitness configuration reliability matrix of the child and the educational behavior of the interactive robot through its internal training parameters and model structure, so as to identify whether the child currently has reasonable mental development and whether it is necessary to output the corresponding mental development reminder signal, so as to promptly notify the guardian to take corresponding measures to avoid potential dangers.

[0064] In summary, the interactive early childhood education system based on motion capture according to the embodiment of the present application is explained, which uses artificial intelligence technology through deep learning to perform data evaluation on the educational instruction information of the interactive robot and the feedback action information of the children, obtains the educational behavior characteristics of the interactive robot and the feedback information characteristics of the children, and characterizes the reliability of the children's mental adaptation through the type feedback interaction characteristics between the two, so as to intelligently judge whether the children have reasonable mental development and provide corresponding development reminders. In this way, it can help guardians to discover potential mental risks in time, so as to take corresponding measures in time to ensure the mental nature of the children's feedback.

[0065] Figure 5 FIG. 1 is a flow chart of an interactive early childhood education method based on motion capture according to an embodiment of the present application. Figure 5 As shown, according to the interactive early childhood education method based on motion capture according to the embodiment of the present application, the steps include: S110, accessing the early childhood feedback action information and interactive robot education instruction information at multiple preset moments within a preset period from the early childhood education cloud platform server, wherein the early childhood feedback action information includes the child's line of sight, facial expression, upper limb action, lower limb action, sound characteristics and interactive feedback, and the interactive robot education instruction information includes voice education instructions, action education instructions and imitation education instructions; S120, performing type-corresponding label allocation on the early childhood feedback action information and the interactive robot education instruction information at the multiple preset moments to obtain a type-corresponding label sequence of early childhood feedback information and a type-corresponding label sequence of interactive robot education behavior; S130, performing type feedback interaction feature acquisition on the type-corresponding label sequence of the early childhood feedback information and the type-corresponding label sequence of the interactive robot education behavior to obtain a child mental adaptation configuration reliability matrix; S140, determining the evaluation record of the child's mental adaptation development through the child mental adaptation configuration reliability matrix.

[0066] The above detailed description of the specific implementation of the invention is only used as an example, and the present application is not limited to the specific implementation described above. For those skilled in the art, any equivalent modification or substitution of the invention is also within the scope of the present application. Therefore, the equal matching, modification, improvement, etc. made without departing from the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. An interactive early childhood education system based on motion capture, characterized in that: include: A preschool education cloud platform data access unit, used to access preschool feedback action information and interactive robot education instruction information at multiple preset moments within a preset period from a preschool education cloud platform server, wherein the preschool feedback action information includes the preschool's sight, facial expression, upper limb action, lower limb action, voice characteristics and interactive feedback, and the interactive robot education instruction information includes voice education instructions, action education instructions and imitation education instructions; A continuous action information corresponding label allocation unit is used to respectively perform type corresponding label allocation on the child feedback action information and the interactive robot education instruction information at the plurality of preset moments to obtain a child feedback information type corresponding label sequence and an interactive robot education behavior type corresponding label sequence; A type feedback interaction unit, used for performing type feedback interaction feature acquisition on the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot educational behavior type to obtain a child mental adaptation confidence matrix; A mental development evaluation unit, used to determine the evaluation record of the child's mental adaptation development through the child's mental adaptation configuration reliability matrix; Wherein, the mental development assessment unit includes: A correction unit, used for performing spatiotemporal dependency restriction of data propagation breadth on the child's mental fitness configuration confidence matrix to obtain a corrected child's mental fitness configuration confidence matrix; The evaluation record output unit is used to obtain the evaluation record by configuring the mental adaptation reliability matrix of the child according to the mental development evaluator through the classifier, and the evaluation record is used to indicate whether to transmit the mental development reminder to the guardian.

2. The interactive early childhood education system based on motion capture as claimed in claim 1, characterized in that: The continuous action information corresponds to a label allocation unit, including: A child feedback information type evaluation unit, configured to arrange the child feedback action information at the plurality of preset moments into a child feedback information input tuple according to time and scene aspects, and then obtain a child feedback information type corresponding label sequence according to a child feedback information type corresponding label distributor using a deep learning model; An interactive robot educational behavior type evaluation unit is used to arrange the interactive robot education instruction information of multiple preset moments into interactive robot education behavior input tuples according to time and scene aspects, and then obtain the interactive robot education behavior type corresponding label sequence based on the education behavior type corresponding label distributor through a deep learning model.

3. The interactive early childhood education system based on motion capture as claimed in claim 2, characterized in that: The type feedback interaction unit includes: A behavior sequence standardization processing unit, used for performing behavior sequence association adjustment on the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot educational behavior type to obtain a label sequence corresponding to the child feedback information type after association and a label sequence corresponding to the interactive robot educational behavior type after association; The feedback prediction unit is used to calculate the feedback prediction of the label sequence corresponding to the child's feedback information type after the association relative to the label sequence corresponding to the interactive robot educational behavior type after the association to obtain the child's mental adaptation confidence matrix.

4. The interactive early childhood education system based on motion capture as claimed in claim 3, characterized in that: The behavior sequence standardization processing unit is used to: The label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot educational behavior type are embedded in the co-space associator through the fully connected layer to obtain the label sequence corresponding to the child feedback information type after association and the label sequence corresponding to the interactive robot educational behavior type after association.

5. The interactive early childhood education system based on motion capture as claimed in claim 4, characterized in that: The feedback prediction unit is used to: The label sequence corresponding to the type of child feedback information after association and the inverse sequence of the label sequence corresponding to the type of interactive robot educational behavior after association are multiplied to obtain the child mental adaptation configuration confidence matrix.

6. The interactive early childhood education system based on motion capture as claimed in claim 5, characterized in that: The correction unit is used for: Calculating the variance of a node group consisting of node values ​​at all positions of the child mental fitness configuration confidence matrix to obtain the node value variance of the child mental fitness configuration confidence matrix; Calculating the square of the absolute value of the node value at each position of the child mental fitness configuration confidence matrix and then performing a likelihood operation to obtain a likelihood matching child mental fitness configuration confidence matrix; Perform cumulative summation on all node values ​​of the likelihood matching child mental fitness configuration confidence matrix to obtain a sum value of the likelihood matching child mental fitness configuration confidence matrix; The sum of the likelihood matching child mental fitness configuration reliability matrix is ​​divided by twice the node value variance of the child mental fitness configuration reliability matrix and then multiplied by the node value of each position of the child mental fitness configuration reliability matrix to obtain the modified child mental fitness configuration reliability matrix.

7. An interactive early childhood education method based on motion capture, characterized in that: include: Accessing the child feedback action information and interactive robot education instruction information at multiple preset moments within a preset period from the early childhood education cloud platform server, wherein the child feedback action information includes the child's sight, facial expression, upper limb action, lower limb action, voice characteristics and interactive feedback, and the interactive robot education instruction information includes voice education instruction, action education instruction and imitation education instruction; Perform type-corresponding label assignment on the child feedback action information and the interactive robot educational instruction information at the plurality of preset moments to obtain a child feedback information type-corresponding label sequence and an interactive robot educational behavior type-corresponding label sequence; Performing type feedback interaction feature acquisition on the label sequence corresponding to the child feedback information type and the label sequence corresponding to the interactive robot educational behavior type to obtain the child mental adaptation confidence matrix; Determine the evaluation record of the child's mental adaptation development through the child's mental adaptation reliability matrix; The evaluation record of the development of children's mental adaptation is determined by the children's mental adaptation reliability matrix, including: Performing spatiotemporal dependency restriction of data propagation breadth on the child's mental fitness configuration confidence matrix to obtain a modified child's mental fitness configuration confidence matrix; The child's mental adaptation reliability matrix is ​​applied to the mental development evaluator through the classifier to obtain the evaluation record, and the evaluation record is used to indicate whether to transmit a mental development reminder to the guardian.

8. The interactive early childhood education method based on motion capture as claimed in claim 7, characterized in that: The type-corresponding label assignment is respectively performed on the child feedback action information and the interactive robot education instruction information at the plurality of preset moments to obtain a child feedback information type-corresponding label sequence and an interactive robot education behavior type-corresponding label sequence, including: Arrange the child feedback action information at the plurality of preset moments according to time and scene aspects into a child feedback information input tuple, and then obtain a label sequence corresponding to the child feedback information type according to a child feedback information type corresponding label distributor using a deep learning model; After arranging the interactive robot education instruction information at multiple preset moments into interactive robot education behavior input tuples according to time and scene aspects, a label distributor corresponding to the education behavior type through a deep learning model is used to obtain a label sequence corresponding to the interactive robot education behavior type.

Images