Preschool education visual interaction method and system

Abstract

The invention relates to the technical field of visual interaction, in particular to a preschool education visual interaction method and system. Comprising the following steps: firstly, collecting historical videos and sound data of a plurality of children, and generating a historical image set and a sound set through preprocessing; and then, marking the data to construct a child state training set, training a child state monitoring model by using the training set, and monitoring the attention state of the current child in real time. And when the attention distraction of the child exceeds a preset time threshold, triggering an attention distraction analyzer to analyze the concentration duration, the basic information and the teaching content, generating an influence vector, generating teaching adjustment suggestion information through an analytic function, and feeding back the teaching adjustment suggestion information to teaching personnel. On the basis, a knowledge base of preschool education experts is combined, personalized teaching adjustment suggestions are generated through an intelligent generation network, and the teaching effect is further improved.

Description

Technical Field

[0001] The present invention relates to the field of visual interaction technology, and in particular to a preschool education visual interaction method and system. Background Art

[0002] Visual interaction refers to the interaction with users through visual elements such as graphics, images, and videos, using visual perception to improve user experience, optimize data analysis, and support the decision-making process. With the continuous advancement of computer technology, visual interaction technology has gradually matured and is widely used in many industries such as medical care, construction, industrial manufacturing, and education.

[0003] In the field of education, especially preschool education, visual interactive technology is significantly changing traditional teaching methods and learning experiences. By closely combining visual elements with learning content, educators can present knowledge in a more vivid and intuitive way, help students understand abstract concepts, and stimulate their interest and enhance their sense of participation. This not only helps to improve learning outcomes, but also enhances children's initiative and interactivity in the learning process.

[0004] As an important part of the education system, preschool education is gradually benefiting from the application of visual interactive technology. Preschool education can not only provide more creative and interesting teaching methods, but also promote children's cognitive development and hands-on ability through interactive games, activities and sensory stimulation. Through advanced technologies such as virtual reality (VR), touch screen devices and interactive charts, teachers can create a learning environment that attracts children's attention and further improve the learning effect of preschool children.

[0005] However, the application of existing visualization interaction technology in preschool education still faces some challenges. Due to their young age, limited attention span, and rapid changes in emotional and cognitive states, preschool children often find it difficult for teachers to detect children's distraction in time in class, which affects the teaching effect. To address this problem, the present invention proposes a visualization interaction method and system for preschool education, which aims to improve the teaching effect by accurately monitoring the children's attention state and providing timely feedback to teachers, and taking effective interactive measures. Summary of the invention

[0006] The purpose of the present invention is to provide a method and system for visual interaction of preschool education. First, the present invention collects historical video data and sound data of children in preschool education, processes these data, and constructs a training set of children's states. In the construction process, an attention state learner is used to label the processed data with state labels. The learner extracts the image features and sound features of children through a feature extraction module, learns various features according to observation information through an attention state setting module, and strengthens abnormal features through an abnormal attention state enhancement module, and finally realizes the attention state labeling of the original data. Secondly, the child state monitoring model is trained using the labeled child state training set. The model is used to monitor the attention state of children in the process of visual teaching, and includes the following layers: input layer, image data processing layer, sound data processing layer, image feature extraction layer, text feature extraction layer, sound feature extraction layer, sub-item area weighting layer, feature merging layer and output layer. The model comprehensively monitors the attention state of children from multiple dimensions by learning image features, text features and sound features. Finally, the present invention proposes an attention distraction analyzer for analyzing the causes of children's attention distraction. The analyzer uses children's personal information, teaching content, and teaching time as influencing variables, calculates the degree of distraction through the attention distraction analysis function, and generates an influencing factor distribution vector. In addition, the present invention also proposes an intelligent suggestion generation network for generating teaching adjustment suggestion information to provide reference for teaching staff so as to better conduct visual interaction.

[0007] To achieve the above object, the present invention provides the following technical solutions: The present invention provides a visual interactive method for preschool education, comprising: obtaining historical video data and historical sound data of multiple children; Further, the historical video data and the historical sound data are preprocessed to generate a historical image set and a historical sound set; wherein the preprocessing process of the historical video data includes: dividing the historical video data according to video frames to obtain multiple first historical images; performing image denoising on multiple first historical images to obtain multiple second historical images; scaling multiple second historical images to a uniform scale to obtain multiple third historical images; performing image enhancement on multiple third historical images to obtain multiple fourth historical images; integrating multiple fourth historical images to obtain the historical image set; the preprocessing process of the historical sound data includes: dividing the historical sound data according to sound intervals to obtain multiple first historical sound data segments; performing sound denoising on multiple first historical sound data segments to obtain multiple second historical sound data segments; performing sound enhancement on multiple second historical sound data segments to obtain multiple third historical sound data segments; integrating multiple third historical sound data segments to obtain the historical sound set.

[0008] Furthermore, the data in the historical image set and the historical sound set are labeled with states, and a child state training set is constructed; wherein an attention state learner is used for state labeling; wherein the attention state learner comprises: A data input module, used for receiving the historical image set and the historical sound set; The information acquisition module includes: an image information acquisition submodule and a sound information acquisition submodule; wherein the image information acquisition submodule is used to acquire the posture information and facial expression information of the child in the historical image set; the sound information acquisition submodule is used to acquire the text information and voice data of the child in the historical sound set; A feature extraction module, used to extract the information features obtained by the image information acquisition submodule and the sound information acquisition submodule to obtain a body posture feature set, a facial feature set, a text feature set and a voice feature set; An attention state setting module, used to analyze the feature extraction results of the feature extraction module and set the attention state for each type of feature according to the observed information; Abnormal attention state enhancement module, used to enhance the features of distracted state; The state labeling module is used to label the state of the data in the historical image set and the historical sound set according to the learned features.

[0009] Furthermore, the child state training set is used to train a child state monitoring model; wherein the training process of the child state monitoring model includes: Input the data samples in the child status training set into the input layer according to the data type; wherein the input layer includes: an image input submodule and a sound input submodule; Furthermore, the image of the image input submodule is input to the image data processing layer to obtain the hand area image, the face area image and the eye area image in the image; Further, the sound data of the sound input submodule is input into the sound data processing layer to obtain text information and sound information; Further, the hand region image, the face region image and the eye region image are input into an image feature extraction layer to obtain a hand feature set, a face feature set and an eye feature set; Further, the text information is input into a text feature extraction layer to obtain a text feature set; Further, the sound information is input into a sound feature extraction layer to obtain a sound feature set; Further, the hand feature set, the face feature set and the eye feature set are input into the sub-item region weighted layer to obtain a hand weighted feature set, a face weighted feature set and an eye weighted feature set; Further, the text feature set and the sound feature set are input into a merging layer to obtain a text-sound feature merged set; Further, the hand weighted feature set, the face weighted feature set, the eye weighted feature set and the text-sound feature combined set are input into an output layer to obtain an attention state monitoring result; wherein the output layer is composed of a classification model; Further, calculating the monitoring error between the attention state monitoring result and the marking result; Furthermore, the child status monitoring model is optimized according to the monitoring error; when the monitoring error reaches convergence, the training of the child status monitoring model is completed.

[0010] Further, the child state monitoring model is used to monitor the current child; when the current child is in a distracted state and the distraction time exceeds a preset time threshold; the current child is used as a target child; Further, the target child is analyzed using an attention distraction analyzer, including: Obtaining the attention span, basic information and teaching content of the target child; Further, information processing is performed on the concentration duration, the basic information and the teaching content to obtain an influence vector; Furthermore, the influence vector is input into the attention distraction analysis function to obtain an analysis vector; wherein the calculation formula of the analysis vector is: ;in, is the analytical vector of the target child; is the dispersion of the target children; is a distribution vector of influencing factors of the target child; is the attention distraction parsing function; is the focus duration vector; is the basic information feature vector; is the characteristic vector of teaching content; Further, generating teaching adjustment suggestion information according to the text information and feeding back to the teaching staff; Wherein, the teaching adjustment suggestion information is generated by using a suggestion intelligent generation network, including: Based on the experience and knowledge of preschool education experts, a preschool education expert knowledge base is constructed; wherein the content of the preschool education expert knowledge base includes: children's teaching cases, teaching coping experience strategies and effect feedback; Further, feature engineering is performed on the content in the preschool education expert knowledge base to obtain a preschool education knowledge feature data set; Further, each piece of data in the preschool education knowledge feature data set is evaluated to obtain a plurality of preschool education knowledge evaluation values; Further, mapping the data in the preschool education knowledge feature data set with the plurality of preschool education knowledge evaluation values ​​to construct a preschool education knowledge graph; Further, obtaining personality description information of the target child and the non-target child; Further, the personality description information is converted into personality description features, and a personality description feature set is constructed; Further, a teaching adjustment suggestion text vector is obtained according to the parsing vector, the character description feature set and the preschool education knowledge graph; Furthermore, the teaching adjustment suggestion text vector is converted into the teaching adjustment suggestion information.

[0011] Furthermore, the teaching staff conducts visual interaction with the children with reference to the teaching adjustment suggestion information.

[0012] A preschool education visualization interactive system, the system comprising: a visualization interface unit, a data acquisition unit, a data storage unit, a data processing unit, a child attention monitoring unit, a child attention distraction analysis unit, a feedback suggestion generation unit, a prompt unit and an interaction unit; Wherein, the visual interface unit is used to display the interactive teaching scene between the teaching staff and the children; The data collection unit is used to collect video data and sound data of children during the learning process; The data storage unit is used to store the data acquired by the data acquisition unit; The data processing unit is used to pre-process the data collected from the children; wherein the data processing unit includes: Acquire historical video data and historical sound data of multiple children in the data storage unit; The preprocessing process of the historical video data includes: dividing the historical video data according to video frames to obtain a plurality of first historical images; performing image denoising on the plurality of the first historical images to obtain a plurality of second historical images; scaling the plurality of the second historical images to a uniform scale to obtain a plurality of third historical images; performing image enhancement on the plurality of the third historical images to obtain a plurality of fourth historical images; integrating the plurality of the fourth historical images to obtain a historical image set; The preprocessing process of the historical sound data includes: dividing the historical sound data according to sound intervals to obtain multiple segments of first historical sound data; performing sound denoising on the multiple segments of the first historical sound data to obtain multiple segments of second historical sound data; performing sound enhancement on the multiple segments of the second historical sound data to obtain multiple segments of third historical sound data; and integrating the multiple segments of the third historical sound data to obtain a historical sound set.

[0013] The child attention monitoring unit is used to monitor the attention state of the child during the learning process; The child attention monitoring unit uses a trained child state monitoring model for monitoring; wherein the training process of the child state monitoring model includes: Input the data samples in the child state training set into the input layer according to the data type; wherein the input layer includes: an image input submodule and a sound input submodule; Furthermore, the image of the image input submodule is input to the image data processing layer to obtain the hand area image, the face area image and the eye area image in the image; Further, the sound data of the sound input submodule is input into the sound data processing layer to obtain text information and sound information; Further, the hand region image, the face region image and the eye region image are input into an image feature extraction layer to obtain a hand feature set, a face feature set and an eye feature set; Further, the text information is input into a text feature extraction layer to obtain a text feature set; Further, the sound information is input into a sound feature extraction layer to obtain a sound feature set; Further, the hand feature set, the face feature set and the eye feature set are input into the sub-item region weighted layer to obtain a hand weighted feature set, a face weighted feature set and an eye weighted feature set; Further, the text feature set and the sound feature set are input into a merging layer to obtain a text-sound feature merged set; Further, the hand weighted feature set, the face weighted feature set, the eye weighted feature set and the text-sound feature combined set are input into an output layer to obtain an attention state monitoring result; wherein the output layer is composed of a classification model; Further, calculating the monitoring error between the attention state monitoring result and the marking result; Furthermore, the child status monitoring model is optimized according to the monitoring error; when the monitoring error reaches convergence, the training of the child status monitoring model is completed.

[0014] The child distraction analysis unit is used to analyze the reasons why the child's attention is distracted; The child attention distraction analysis unit uses an attention distraction analyzer to perform attention distraction analysis; wherein the attention distraction analyzer includes: According to the monitoring result of the child attention monitoring unit, the child with distracted attention is regarded as the target child; Furthermore, the concentration time, basic information and teaching content of the target child are obtained; wherein the basic information includes: age, gender and personality description; Further, information processing is performed on the concentration duration, the basic information and the teaching content to obtain an influence vector; Furthermore, the influence vector is input into the attention distraction analysis function to obtain an analysis vector; wherein the calculation formula of the analysis vector is: ;in, is the analytical vector of the target child; is the dispersion of the target children; is a distribution vector of influencing factors of the target child; is the attention distraction parsing function; is the focus duration vector; is the basic information feature vector; is the characteristic vector of teaching content; The feedback suggestion generating unit is used to generate teaching adjustment suggestion information; wherein the feedback suggestion generating unit uses a suggestion intelligent generating network to generate the teaching adjustment suggestion information, and the process includes: Based on the experience and knowledge of preschool education experts, a preschool education expert knowledge base is constructed; wherein the content of the preschool education expert knowledge base includes: children's teaching cases, teaching coping experience strategies and effect feedback; Further, feature engineering is performed on the content in the preschool education expert knowledge base to obtain a preschool education knowledge feature data set; Further, each piece of data in the preschool education knowledge feature data set is evaluated to obtain a plurality of preschool education knowledge evaluation values; Further, mapping the data in the preschool education knowledge feature data set with the plurality of preschool education knowledge evaluation values ​​to construct a preschool education knowledge graph; Further, obtaining personality description information of the target child and the non-target child; Further, the personality description information is converted into personality description features, and a personality description feature set is constructed; Further, according to the parsing vector, the character description feature set and the preschool education knowledge graph, a teaching adjustment suggestion text vector is obtained; Further, converting the teaching adjustment suggestion text vector into the teaching adjustment suggestion information; The prompting unit is used to remind the teaching staff of the current status of the child; The interactive unit is used for visual interaction between teaching staff and children.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention proposes an attention state learner for annotating the states of historical video data and historical sound data of preschool education visualization children, and constructing a child state training set; the attention state learner includes multiple modules: a data input module receives image and sound data, an information acquisition module extracts the child's posture, facial expression, voice and text features respectively, a feature extraction module further analyzes these features, an attention state setting module calibrates the attention state based on the extraction results, an abnormal state enhancement module is used to enhance the scattered state features, and finally the data is annotated through the state annotation module to construct an accurate child state training set, which promotes the accuracy of the child state monitoring model monitoring; it is conducive to subsequent visualization interaction and improves the teaching quality.

[0016] 2. The present invention proposes a child status monitoring model, which inputs, extracts features, weights and merges multimodal data of child status (including images, sounds, and text), and finally obtains the attention status monitoring results through the output layer, and optimizes the child status monitoring model based on the monitoring error. This method can accurately identify the attention status of children and improve the monitoring accuracy by continuously optimizing the model. For preschool education visual interaction, the system can monitor and analyze children's attention changes in real time, help teachers adjust teaching strategies in a timely manner, and provide targeted support, thereby improving the quality of classroom interaction and teaching effectiveness.

[0017] 3. The present invention proposes an attention distraction analyzer for analyzing when children are distracted; the attention distraction analyzer analyzes the target child through the attention distraction analyzer to obtain the child's concentration duration, basic information and teaching content, and obtains the influence vector through information processing. The influence vector is input into the attention distraction analysis function, and the analysis vector is calculated to further quantify the child's degree of attention distraction. In preschool education, visual interaction can analyze children's attention distraction in real time, help teachers identify factors that affect concentration and learning effects, and improve the quality of classroom interaction by accurately understanding each child's concentration duration and the factors affected.

[0018] 4. The present invention proposes a suggestion intelligent generation network that combines the experience knowledge base, personality description information and preschool education knowledge map of preschool education experts through an intelligent generation network to provide personalized teaching adjustment suggestions for teaching staff. The specific steps include building an expert knowledge base and performing feature engineering, evaluating and mapping knowledge features, combining the personality characteristics of the target children, and finally generating a teaching adjustment suggestion text. Teaching staff can refer to these suggestions to optimize interaction with children and improve the quality of classroom teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a visual interactive method for preschool education provided by an embodiment of the present invention; Figure 2 A structural diagram of a preschool education visual interactive system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Visual interactive technology interacts with users through graphics, images, and video visual elements to enhance user experience through visual perception. Visual interactive technology has been applied to many industries such as medicine, construction, and education. In the field of education, especially preschool education, visual interactive technology is significantly changing traditional teaching methods and learning experiences. By closely combining visual elements with learning content, educators can present knowledge in a more vivid and intuitive way, help students understand abstract concepts, and stimulate their interest and sense of participation. This not only helps to improve learning outcomes, but also enhances children's initiative and interactivity in the learning process. However, in this process, preschool children are young, have limited attention span, and their emotions and cognitive states change rapidly; using existing technologies, teachers often find it difficult to detect children's distraction in time in class. Therefore, in response to the above problems, the present invention proposes a visual interactive method and system for preschool education; in order to illustrate the implementation effect of the present invention, the following two embodiments will be described in detail;

[0022] Embodiment 1: In the embodiment of the present application, the method and system proposed in the present invention are used to describe in detail the visualization process of preschool education. It is known that in the embodiment of the present application, preschool education is for children aged 3 to 6 years old, with a total of 10 children participating and learning through online videos; Figure 1and Figure 2 Content: Preschool children can conduct visual interaction process description; among them, Figure 1 The specific process of the method proposed in the present invention includes: S10. Acquire historical video data and historical sound data of multiple children; S20. Preprocess the historical video data and the historical sound data to generate a historical image set and a historical sound set; S30. Train a child status monitoring model; S40. Use the child status monitoring model to monitor the current child; S50. Obtain the target child based on the monitoring results; S60. Use an attention distraction analyzer to analyze the target child; S70. Generate teaching adjustment suggestion information based on the analytical vector and feedback it to the teaching staff; S80. The teaching staff refers to the teaching adjustment suggestion information to conduct visual interaction with the children. Figure 2 The structural diagram of the system proposed by the present invention includes: a visual interface unit, a data acquisition unit, a data storage unit, a data processing unit, a child attention monitoring unit, a child attention distraction analysis unit, a feedback suggestion generation unit, a prompt unit and an interaction unit; combined with Figure 1 and Figure 2 The contents in are described as follows; The historical video data and historical sound data of the 10 children are obtained from the data storage unit of the system, corresponding to the above step S10; wherein the above historical video data and historical sound data are obtained from the learning process of the 10 children in the visual interface unit through the data acquisition unit of the system. The data acquisition unit collects the video and sound data of the learning process in real time based on the first monitoring position, the second monitoring position and the microphone in the learning device used by the children.

[0023] Further, the historical video data and the historical sound data are preprocessed by the data processing unit of the system to obtain a historical image set and a historical sound set; corresponding to step S20; wherein the specific implementation process of the data processing unit includes: The preprocessing process of the historical video data includes: dividing the historical video data according to video frames to obtain multiple first historical images; performing image denoising on the multiple first historical images to obtain multiple second historical images; scaling the multiple second historical images to a uniform scale to obtain multiple third historical images; performing image enhancement on the multiple third historical images to obtain multiple fourth historical images; integrating the multiple fourth historical images to obtain the historical image set; the preprocessing process of the historical sound data includes: dividing the historical sound data according to sound intervals to obtain multiple segments of first historical sound data; performing sound denoising on multiple segments of the first historical sound data to obtain multiple segments of second historical sound data; performing sound enhancement on multiple segments of the second historical sound data to obtain multiple segments of third historical sound data; integrating multiple segments of the third historical sound data to obtain the historical sound set.

[0024] In the application embodiment, the data processing unit is used to preprocess the historical video data and historical sound data of the child to obtain a historical image and historical sound set; by preprocessing the historical video and sound data (including denoising, scaling, enhancement, etc.), the noise and irrelevant information in the original data can be effectively removed, and the important behavior and emotional signals of the child in the learning process can be retained. These high-quality data will become the basis for subsequent training of child status monitoring, and will help improve the model's ability to perceive and recognize changes in children's attention.

[0025] Furthermore, the historical image set and the historical sound set are annotated to construct a child state training set. The attention state learner is used to annotate the state of the above historical data. The specific process is as follows: Inputting the historical image set and the historical sound set into a data input module; Further, the information in the historical image set and the historical sound set is obtained by using the information acquisition module; wherein the information acquisition module includes: an image information acquisition submodule and a sound information acquisition submodule; The image information acquisition submodule is used to obtain the body posture information and facial expression information of the children in the historical image set; including: analyzing the body posture of the children in the image (for example, whether they are sitting upright, whether their heads are kept straight ahead, etc.) through image processing algorithms; and extracting the facial area of ​​the children using facial recognition technology.

[0026] The sound information acquisition submodule is used to acquire text information and voice data of the children in the historical image set; including: using speech recognition technology (such as automatic speech recognition system, ASR) to transcribe the processed speech signal into text; Furthermore, the feature extraction module extracts features from the information obtained in the image information submodule and the sound information acquisition submodule to obtain a body posture feature set, a facial feature set, a text feature set, and a voice feature set; The acquisition process of the posture feature set, the facial feature set, the text feature set and the voice feature set includes: (1) Body posture feature set: Input data: Image data from the image information acquisition submodule, containing full-body images of children.

[0027] Processing method: Use human pose estimation technology (such as OpenPose, HRNet, etc.) to detect key points of the human body in the image and extract the coordinate information of important joints such as the head, shoulders, arms, elbows, knees and feet.

[0028] Feature extraction: By normalizing and vectorizing the coordinates of key points of the human body, a set of posture features is constructed, including information such as the human body's static posture, dynamic movements, direction of limb movement, angle changes, etc.

[0029] Output: A set of body posture features, including key point coordinates, motion trajectory and other information.

[0030] (2) Facial feature set: Input data: face images from the image information acquisition submodule.

[0031] Processing method: Use facial recognition or facial key point detection technology (such as Dlib, MTCNN, etc.) to process the child’s face in the image and extract the locations of facial key points, including eyes, nose, mouth, eyebrows, etc.

[0032] Feature extraction: By extracting facial geometric features (such as facial contours, eye openness, mouth opening, etc.), or using facial expression recognition models to further extract information such as emotional state.

[0033] Output: Facial feature set: including facial key points, expression features, etc.

[0034] (3) Text feature set: Input data: speech transcription text from the sound information acquisition submodule; Processing method: Use natural language processing (NLP) technology to process text, including word segmentation, word vector representation (such as Word2Vec, BERT, etc.), word frequency statistics, sentiment analysis, etc.

[0035] Feature extraction: Extract semantic information, sentiment, keywords, and syntactic structures from the text, and further extract the potential meaning and information of the text through context analysis.

[0036] Output: A collection of text features, including word embeddings, sentiment scores, and keywords.

[0037] (4) Speech feature set: Input data: sound data from the sound information acquisition submodule.

[0038] Processing method: speech feature extraction (such as MFCC, spectrogram, pitch and volume, etc.).

[0039] Feature extraction: Extract time domain and frequency domain features from sound signals.

[0040] Output: A collection of speech features, including pitch, rhythm, volume, speaking rate, and sentiment analysis; Furthermore, the attention state setting module is used to analyze the features in the posture feature set, the facial feature set, the text feature set and the voice feature set, and the attention state of each feature is set according to the previous observation information; wherein the attention state includes: concentration and distraction; The specific process of the attention state setting module includes: (1) Input a body feature set, a facial feature set, a text feature set, and a speech feature set; (2) Characterize the “distracted” information in previous observations; (3) Feature analysis: Use machine learning or deep learning algorithms to analyze the relevance and influence of features such as posture, facial expressions, text, and voice.

[0041] (4) Global weight calculation of distracted features: The system combines historical observation information and current input data to calculate the weight of each feature in the distracted state.

[0042] (5) Feature fusion: Features from different modalities are fused, and weighted sum, weighted average, or attention mechanism in deep learning may be used to enhance the contribution of each feature.

[0043] (6) Attention state setting: Based on the contribution of each feature, if the subject tends to be distracted (e.g., loose posture, unstable voice tone, bored or sleepy facial expression, etc.), the attention state is set to "distracted". On the contrary, if the subject shows highly concentrated and focused behavior (e.g., stable posture, focused facial expression, clear and organized voice), the attention state is set to "focused".

[0044] Furthermore, the attention distraction feature is input into the abnormal attention state enhancement module for feature enhancement; wherein the feature enhancement formula of the abnormal attention state enhancement module is: ;in, For the enhanced distraction feature; for unenhanced distraction characteristics; is the distraction feature enhancement coefficient; Furthermore, the state of the original data samples with enhanced attention distraction features is annotated through a state annotation module.

[0045] In the embodiment of the present application, an attention state learner is used to annotate the historical data of children; the attention state learner is based on the learning and annotation of children's attention states of multimodal data (historical image set and historical sound set). Its main goal is to construct a training set of children's attention states by analyzing children's posture, facial expressions, text information and voice data, and perform data processing and feature extraction through multiple modules to achieve accurate annotation of attention states.

[0046] Further, the child state training set is used to train the child state monitoring model of the child attention monitoring unit in the system, corresponding to step S30; the specific process of training the child state monitoring model includes: (1) Data input: Objective: Input the data samples in the child state training set into the input layer according to the data type.

[0047] Operation: The input layer consists of two submodules: image input submodule and sound input submodule, which process image and sound data respectively. The input data of each module will be classified and processed according to different data types.

[0048] (2) Image data processing: Objective: Extract the hand, face, and eye regions from an image.

[0049] Operation: The image input submodule passes the input image data to the image data processing layer, where the image is cut or extracted into different regions (hands, face, and eyes), which are the key areas related to children's attention monitoring.

[0050] (3) Sound data processing: Objective: Extract text and sound information from sound data.

[0051] Operation: The sound input submodule passes the sound data to the sound data processing layer, which is responsible for converting the sound signal into text information (such as speech recognition results) and sound information (such as volume, speaking speed, etc.). This information will serve as the basis for subsequent feature processing.

[0052] (4) Image feature extraction: Objective: Extract a set of features from images of hand, face, and eye regions.

[0053] Operation: The extracted hand region images, face region images, and eye region images are respectively input into the image feature extraction layer, which uses computer vision technology (such as convolutional neural network) to extract the feature set of each region. These features will be used for subsequent weighted processing.

[0054] (5) Text feature extraction: Objective: Extract feature sets from text information.

[0055] Operation: Input the text information obtained from the sound processing layer to the text feature extraction layer to extract the feature set of the text. These features include semantic information, sentiment analysis results, etc., which are helpful for evaluating the attention status of children.

[0056] (6) Sound feature extraction: Goal: Extract feature sets from sound information.

[0057] Operation: Input the sound data (such as speech rate, voice emotion, and pitch) into the sound feature extraction layer to extract the relevant features of the sound. These features reflect the characteristics of the child's sound signal and help determine the child's attention level.

[0058] (7) Feature weighting: Goal: To weight different sets of regional features to better reflect the importance of each region.

[0059] Operation: Input the feature set of the hand, face and eye into the sub-region weighted layer to calculate the weighted feature set. This weighting process will be adjusted according to the importance of each region to enhance the weight of useful features.

[0060] (8) Feature merging: Goal: Fuse text features and sound features to create a unified feature set.

[0061] Operation: Input the text feature set and the sound feature set into the feature merging layer. In this layer, the text and sound information will be merged into a unified feature set to provide more comprehensive information about the child's status.

[0062] (9) Output layer processing: Objective: To output the monitoring results of children’s attention status through classification model.

[0063] Operation: The weighted image feature set (hands, face, eyes) and the combined text-sound feature set are input to the output layer. The output layer is a classification model, which is a CNN classification model; based on these input features, the child's attention state is classified (such as "focused", "distracted", etc.).

[0064] (10) Error calculation and optimization: Objective: Calculate the error between the monitoring results and the annotation results, and optimize the model.

[0065] Operation: Calculate the monitoring error by comparing the attention state monitoring results output by the model with the actual labeled results. Based on the error, use common optimization algorithms (such as back propagation) to optimize the child state monitoring model to improve the accuracy of the model.

[0066] (11) Model convergence and training completion: Goal: When the monitoring error reaches the convergence criterion, the training is completed.

[0067] Operation: Through iterative optimization, the monitoring error is gradually reduced. The training process will terminate when the error reaches the predetermined convergence standard, indicating that the training of the child state monitoring model is completed and the model can better predict the child's attention state. Refer to Table 1, which shows the changes in various performance indicators during the training process of the child state monitoring model; Table 1 Description of the training performance of the child status monitoring model

[0068] In Table 1, the accuracy trend: starting from 0.20 in the initial stage, the accuracy gradually improved with the increase of training rounds; in the 10th round of training: the accuracy increased to 0.60, indicating that the model has made significant progress after 10 rounds of training. In the 100th round of training: the accuracy further increased to 0.93, and finally in the test stage, the accuracy reached 0.95, showing the learning and optimization effect of the model. Training loss trend: the training loss decreased from 1.45 in the initial stage to 0.09 in the final model, showing that the model gradually reduced the error as the training deepened, showing good fitting ability. Round 10: the training loss was 0.92, although it decreased, it was still high. Round 100: the loss dropped to 0.12, close to the minimum value, indicating that the model basically converged. Validation loss trend: the validation loss decreased from 1.42 in the initial stage to 0.11, indicating that the model is not only gradually optimized on the training data, but also shows better generalization ability on the validation set. Round 10: Validation loss is 0.88, which is lower than the training loss, but still has some error. Round 100: Validation loss further drops to 0.15, showing good generalization ability. F1 score trend: The F1 score increases from 0.18 in the initial stage to 0.94 in the final model, showing that as the training progresses, the model's ability to balance precision and recall gradually increases. Round 10: The F1 score is 0.58, which is a significant improvement, but there is still a certain gap compared with the final model. Round 100: The F1 score further increases to 0.92, and the F1 score of the final model is 0.94, indicating that the model is more accurate in predicting positive samples. Precision trend: The precision increases from 0.15 in the initial stage to 0.93 in the final model, indicating that the model has significantly improved its accuracy in predicting positive samples. Round 10: The precision is 0.55, indicating that the model still has some error in predicting positive samples. Round 100: Precision increased to 0.91, and the final model had a precision of 0.93, which is excellent. Recall trend: Recall increased from 0.12 in the initial stage to 0.95 in the final model, indicating that the model has made significant improvements in identifying positive samples. Round 10: Recall was 0.62, indicating that the model has made progress in capturing positive samples. Round 100: Recall increased to 0.93, and the final model had a recall of 0.95, showing the model's high recognition ability for positive samples.

[0069] In an embodiment of the present application, a child status training set is used to train a child status monitoring model; wherein, the child status monitoring model can be analyzed from multiple aspects, especially in a training model based on multimodal data (such as images, sounds, texts, etc.), which can provide a more accurate and comprehensive child status assessment.

[0070] Further, the trained child status monitoring model is used to monitor the 10 children currently being studied in the embodiment of the present application, corresponding to step S40; Furthermore, according to the monitoring results of the child status monitoring model, the attention status of 10 children is obtained; when the attention is in a distracted state and the distraction time exceeds the preset time threshold; the corresponding child is used as the target child, corresponding to step S50; wherein, in the implementation of this application, the threshold is set to different time lengths according to different system designs. Common settings are as follows: (1) 10 seconds to 30 seconds: For younger children or distracted tasks in a short time, it will be set to 10 seconds to 30 seconds. (2) 30 seconds to 1 minute: For situations with more complex tasks or longer learning content, a longer threshold time is required, such as 30 seconds to 1 minute. (3) More than 1 minute: For long-term distraction, the threshold is set to 1 minute or longer, especially in complex learning tasks.

[0071] Furthermore, the prompt unit of the system displays the monitoring results through the visualization unit, and displays the current status of the 10 children in real time; the teaching staff can process the results displayed on the visualization interface; when the number of target children is small, the teaching staff can interactively remind the target children; when the number of target children is large, the child attention distraction analysis unit of the system performs analysis; from the display information of the visualization unit, it can be seen that there are 7 target children at present; refer to Table 2, which shows the status information of the 10 children; Table 2 Status information of 10 children

[0072] Further, the target children in Table 2 are analyzed using the child distraction analysis unit, corresponding to step S60; wherein the child distraction analysis unit uses a child distraction analyzer for analysis, and the specific process includes: Obtain the attention span, basic information, and teaching content of the seven target children; the attention span refers to the time interval from the last time the child focused to the time point when the child is currently assessed as a target child; the basic information refers to the basic information such as the child’s age, gender, and personality; the teaching content refers to the tasks or learning materials that the child is learning, including its difficulty, learning objectives, and content type; Furthermore, the attention duration, basic information and teaching content are processed to obtain the influence vector; wherein the information processing of the above contents includes: (1) The standardized processing of focus duration is: ; (2) Processing of basic information (age, gender, and personality); Age, as a numerical feature, can be standardized. For example, if the preschool age range is 3 to 6 years old, we can standardize as follows: ; Gender is usually a categorical feature and can be processed by "one-hot encoding". For the gender variables "male" and "female", it can be converted to: ; ; Personality traits can also be encoded in a similar way. Assuming that personality is divided into two categories: extroversion and introversion: ; ; Teaching content can also be processed by numerical coding. Since the teaching content of all target children is the same in this case, only one dimension is needed to represent this feature. The teaching content can be assigned a value (such as 1), or further refined according to the difficulty, type, and other dimensions of the content. See Table 3 for the impact vectors after information processing for the seven target children; Table 3 Influence vectors on target children

[0073] Furthermore, the influence vector is input into the attention distraction analysis function to obtain an analysis vector; wherein the calculation formula of the analysis vector is: ;in, is the analytical vector of the target child; is the dispersion of the target children; is a distribution vector of influencing factors of the target child; is the attention distraction parsing function; is the focus duration vector; is the basic information feature vector; is the characteristic vector of teaching content; Taking Child 1 as an example, the calculation process of its analytical vector is: Step 1: Calculate the dispersion; Distraction is usually related to the degree of distraction of the child, and the duration of distraction for each child (in seconds) has been provided. For example, Child 1's distraction time was 45 seconds, which exceeded the preset threshold of 10 seconds, so he was classified as a target child. As mentioned in Table 2, distraction is related to the severity of the attention state. According to the length of the distraction time, a numerical distraction can be defined; the value of the distraction is proportional to the duration of the distraction. For example: Equal distraction: corresponding to a distraction time between 10 seconds and 50 seconds (such as children 1, 4, 7 and 10); Severe distraction: corresponding to a distraction time of more than 50 seconds (such as children 2, 6 and 9).

[0074] Step 2: Calculate the distribution vector of influencing factors of child 1; The influence vector of child 1 is: [0.625, 0.6667, 1, 0, 1], and the dispersion is medium (weight is 0.5). Then the influence factor distribution vector can be calculated by the following steps: ; Step 3: Combining the dispersion and the influencing factor distribution vector to obtain the analytical vector; The analytic vector for Child 1 is: ; See Table 4, which gives the analytic vectors of the target children; Table 4. Analysis vectors of target children

[0075] Furthermore, the feedback suggestion generation unit of the system is used to generate teaching adjustment suggestion information according to the analytical vector of the target child, and the information is fed back to the teaching staff, corresponding to step S70; wherein the feedback suggestion generation unit adopts a suggestion intelligent generation network, and the specific process includes: Based on the experience and knowledge of preschool education experts, a preschool education expert knowledge base is constructed; wherein the content of the preschool education expert knowledge base includes: children's teaching cases, teaching response experience strategies and effect feedback; wherein, children's teaching cases are the collection of specific teaching examples, including the behavior performance, learning status and teacher's response measures of children of different ages. Teaching response experience strategies are how teachers adjust teaching methods according to different situations, such as how to deal with children's inattention and behavioral problems. Effect feedback is the effect evaluation after teaching adjustment, including children's learning progress and emotional changes.

[0076] Furthermore, feature engineering is performed on the content in the preschool education expert knowledge base to obtain a preschool education knowledge feature data set; feature engineering is to convert the original data into a format that can be used for machine learning and analysis. In preschool education, the goal of feature engineering is to extract key features that can describe teaching effects and children's behaviors.

[0077] Extract features from teaching cases: for example, children’s response types (positive, negative, no response), application of teaching strategies (such as storytelling, interactive games, reward and punishment mechanisms, etc.), teachers’ feedback methods (verbal praise, behavioral rewards, emotional support, etc.).

[0078] Extract features from empirical strategies: such as the teaching strategies adopted by teachers and the applicable scenarios and goals of the strategies.

[0079] Extract features from performance feedback: for example, learning progress, emotional state (anxious, happy, irritable), social skills, etc.

[0080] Further, each piece of data in the preschool education knowledge feature data set is evaluated to obtain a plurality of preschool education knowledge evaluation values; wherein the preschool education knowledge evaluation value gradient boosting tree is evaluated from multiple dimensions; Furthermore, the data in the preschool education knowledge feature data set is mapped with a plurality of preschool education knowledge evaluation values ​​to construct a preschool education knowledge graph; including: Node definition: Each node in the graph represents a specific knowledge unit, which can be a teaching strategy, teaching case, children's behavior pattern, teacher's feedback method, etc.

[0081] Edge definition: The edges in the graph represent the relationship between different knowledge units, such as "teaching strategy A is applicable to behavioral performance B", "emotional state C is related to learning progress D", etc.

[0082] Multi-dimensional information integration: Integrate information of different dimensions obtained from the knowledge base of preschool education experts into the knowledge graph. Each node does not only represent a single concept, but contains multi-dimensional data and evaluation information. For example, the node "reward mechanism" can contain information such as the applicable scenarios of the strategy, implementation methods, children's response types, and learning effect evaluation.

[0083] Dynamic update and optimization: The preschool education knowledge map is an evolving process. With the continuous addition of new teaching cases, teacher feedback, children's performance and other data, the map will be continuously updated and optimized; Further, obtaining personality description information of the target children and non-target children; wherein the non-target children refer to children whose attention state is focused during the monitoring process; Further, the personality description information is converted into personality description features, and a personality description feature set is constructed; keywords describing personality can be identified through keyword extraction technology (such as TF-IDF), and further matched with personality features; Furthermore, according to the parsing vector, the character description feature set and the preschool education knowledge graph, a teaching adjustment suggestion text vector is obtained; the process includes: A matching model based on deep learning matches students’ comprehensive feature vectors with the teaching strategies recommended in the preschool education knowledge graph; Based on students’ personality traits and behavioral data, infer the most appropriate teaching adjustment strategy; Further, the teaching adjustment suggestion text vector is converted into teaching adjustment suggestion information; wherein the teaching adjustment suggestion text vector is converted into natural language text by natural language generation (NLG) technology; In the embodiment of the present application, a suggestion intelligent generation network is used to provide personalized teaching adjustment suggestions for teaching staff by combining the experience knowledge base, personality description information and preschool education knowledge graph of preschool education experts through the intelligent generation network; teaching staff can refer to these suggestions to optimize interaction with children and improve the quality of classroom teaching.

[0084] Furthermore, through the interactive unit of the system, the teaching staff refers to the teaching adjustment suggestion information to conduct visual interaction with the children, corresponding to step S80.

[0085] In the embodiment of the present application, the method and system of the present invention realize accurate state monitoring and real-time attention distraction analysis of 10 children. By collecting historical video data and sound data of children in the learning process, and performing a series of preprocessing operations on these data, including denoising, enhancement and scale unification, a historical image set and a historical sound set of each child are constructed. These data sets are accurately labeled with states to form a comprehensive child state training set for training the child state monitoring model.

[0086] The model combines image data (such as children's facial expressions, body posture, eye and hand movements, etc.) with sound data (including voice, intonation, speaking speed, etc.) to effectively monitor the changes in children's various attention states during the learning process. In particular, when a child is detected to be distracted and exceeds a preset time threshold, the system can identify the child in real time and mark the child as a target child.

[0087] Furthermore, using the attention distraction analyzer, the system analyzes the target children's attention span, basic information (such as age, gender, personality traits, etc.) and the teaching content they participate in, generates an influence vector and analyzes it through the attention distraction analysis function to obtain an analytical vector. These analytical vectors not only help determine the specific reasons for children's attention distraction, but also provide specific teaching adjustment suggestions for teaching staff, thereby optimizing teaching strategies.

[0088] On this basis, it is suggested that the intelligent generation network further optimizes teaching suggestions by integrating the experience knowledge base of preschool education experts and combining specific teaching cases and response strategies. Through the construction of the preschool education knowledge map, the system can generate personalized teaching adjustment plans based on the personality characteristics, learning environment and historical data of the target children. Finally, the teaching staff can interact with the children visually based on the adjustment suggestion information fed back by the system to improve the educational effect.

[0089] Embodiment 2: In Example 1, the method and system proposed in the present invention successfully implemented the visual interactive suggestion process for 10 children when their attention was distracted. To further verify the effectiveness of the present invention, the application experiment was also conducted on other online learning classes in preschool education tasks in the embodiment of this application. In the experiment, a total of 10 children participated, including 4 girls and 6 boys. The specific process is as follows: Obtain historical video data and historical sound data of 10 children in the application embodiment; Further, preprocessing the historical video data and the historical sound data to generate a historical image set and a historical sound set; Further, the data in the historical image set and the historical sound set are labeled with states, and a child state training set is constructed; wherein the state labeling is performed using an attention state learner; Furthermore, the child state monitoring model is trained using the child state training set; Further, the trained child status monitoring model is used to monitor 10 children in the embodiment of the present application; when the monitoring result of the child status monitoring model is in a dispersed state and the dispersed time exceeds the preset time threshold, the target child is obtained; Further, the target child is analyzed using an attention distraction analyzer to obtain a parsing vector; Furthermore, a suggestion intelligent generation network is used to generate teaching adjustment suggestion information, including: Based on the experience and knowledge of preschool education experts, a preschool education expert knowledge base is constructed; wherein the content of the preschool education expert knowledge base includes: children's teaching cases, teaching coping experience strategies and effect feedback; Further, feature engineering is performed on the content in the preschool education expert knowledge base to obtain a preschool education knowledge feature data set; Further, each piece of data in the preschool education knowledge feature data set is evaluated to obtain a plurality of preschool education knowledge evaluation values; Further, mapping the data in the preschool education knowledge feature data set with the plurality of preschool education knowledge evaluation values ​​to construct a preschool education knowledge graph; Further, obtaining personality description information of the target child and the non-target child; Further, the personality description information is converted into personality description features, and a personality description feature set is constructed; Further, a teaching adjustment suggestion text vector is obtained according to the parsing vector, the character description feature set and the preschool education knowledge graph; Furthermore, the teaching adjustment suggestion text vector is converted into the teaching adjustment suggestion information.

[0090] Furthermore, the teaching staff conducts visual interaction with the children with reference to the teaching adjustment suggestion information.

[0091] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A visual interactive method for preschool education, characterized in that: include: Acquire historical video data and historical sound data of multiple children; pre-process the historical video data and the historical sound data to generate a historical image set and a historical sound set; annotate the state of the data in the historical image set and the historical sound set, and construct a child state training set; The child status training set is used to train a child status monitoring model; the child status monitoring model is used to monitor the current child; When the child is currently in a distracted state and the distraction time exceeds a preset time threshold; taking the current child as a target child; The target child is analyzed by using an attention distraction analyzer, including: obtaining the attention duration, basic information and teaching content of the target child; performing information processing on the attention duration, the basic information and the teaching content to obtain an influence vector; inputting the influence vector into an attention distraction analysis function to obtain an analysis vector; wherein the calculation formula of the analysis vector is: ;in, is the analytical vector of the target child; The dispersion of the target children; is a distribution vector of influencing factors of the target child; is the attention distraction parsing function; is the focus duration vector; is the basic information feature vector; is the characteristic vector of teaching content; Teaching adjustment suggestion information is generated according to the analytical vector and fed back to the teaching staff; the teaching staff conducts visual interaction with the children with reference to the teaching adjustment suggestion information.

2. A visual interactive method for preschool education according to claim 1, characterized in that: The preprocessing process of the historical video data includes: dividing the historical video data according to video frames to obtain multiple first historical images; performing image denoising on the multiple first historical images to obtain multiple second historical images; scaling the multiple second historical images to a uniform scale to obtain multiple third historical images; performing image enhancement on the multiple third historical images to obtain multiple fourth historical images; integrating the multiple fourth historical images to obtain the historical image set; the preprocessing process of the historical sound data includes: dividing the historical sound data according to sound intervals to obtain multiple segments of first historical sound data; performing sound denoising on multiple segments of the first historical sound data to obtain multiple segments of second historical sound data; performing sound enhancement on multiple segments of the second historical sound data to obtain multiple segments of third historical sound data; integrating multiple segments of the third historical sound data to obtain the historical sound set.

3. A visual interactive method for preschool education according to claim 1, characterized in that: An attention state learner is used for state labeling; wherein the attention state learner comprises: A data input module, used for receiving the historical image set and the historical sound set; The information acquisition module includes: an image information acquisition submodule and a sound information acquisition submodule; wherein the image information acquisition submodule is used to acquire the posture information and facial expression information of the child in the historical image set; the sound information acquisition submodule is used to acquire the text information and voice data of the child in the historical sound set; A feature extraction module, used to extract the information features obtained by the image information acquisition submodule and the sound information acquisition submodule to obtain a body posture feature set, a facial feature set, a text feature set and a voice feature set; An attention state setting module, used to analyze the feature extraction results of the feature extraction module and set the attention state for each type of feature according to the observed information; Abnormal attention state enhancement module, used to enhance the features of distracted state; The state labeling module is used to label the state of the data in the historical image set and the historical sound set according to the learned features.

4. A visual interactive method for preschool education according to claim 1, characterized in that: The training process of the child status monitoring model includes: Input the data samples in the child status training set into the input layer according to the data type; wherein the input layer includes: an image input submodule and a sound input submodule; Inputting the image of the image input submodule to the image data processing layer to obtain the hand area image, the face area image and the eye area image in the image; Inputting the sound data of the sound input submodule into the sound data processing layer to obtain text information and sound information; Inputting the hand region image, the face region image and the eye region image into an image feature extraction layer to obtain a hand feature set, a face feature set and an eye feature set; Inputting the text information into a text feature extraction layer to obtain a text feature set; Inputting the sound information into a sound feature extraction layer to obtain a sound feature set; Inputting the hand feature set, the face feature set and the eye feature set to the sub-item region weighted layer to obtain a hand weighted feature set, a face weighted feature set and an eye weighted feature set; Inputting the text feature set and the sound feature set into a merging layer to obtain a text-sound feature merging set; Inputting the hand weighted feature set, the face weighted feature set, the eye weighted feature set and the text-sound feature combined set into an output layer to obtain an attention state monitoring result; wherein the output layer is composed of a classification model; Calculating a monitoring error between the attention state monitoring result and the labeling result; The child status monitoring model is optimized according to the monitoring error; when the monitoring error reaches convergence, the training of the child status monitoring model is completed.

5. The method for visualizing and interacting with preschool education according to claim 1, characterized in that: The teaching adjustment suggestion information is generated by using a suggestion intelligent generation network, including: constructing a preschool education expert knowledge base based on the experience and knowledge of preschool education experts; wherein the content of the preschool education expert knowledge base includes: children's teaching cases, teaching response experience strategies and effect feedback; performing feature engineering on the content in the preschool education expert knowledge base to obtain a preschool education knowledge feature data set; evaluating each data in the preschool education knowledge feature data set to obtain multiple preschool education knowledge evaluation values; mapping the data in the preschool education knowledge feature data set with the multiple preschool education knowledge evaluation values ​​to construct a preschool education knowledge graph; obtaining the personality description information of the target children and non-target children; converting the personality description information into personality description features and constructing a personality description feature set; obtaining a teaching adjustment suggestion text vector based on the analytical vector, the personality description feature set and the preschool education knowledge graph; and converting the teaching adjustment suggestion text vector into the teaching adjustment suggestion information.

6. A preschool education visual interactive system, characterized in that: The system includes: a visualization interface unit, a data acquisition unit, a data storage unit, a data processing unit, a child attention monitoring unit, a child attention distraction analysis unit, a feedback suggestion generation unit, a prompt unit and an interaction unit; wherein the visualization interface unit is used to display the interactive teaching scene between teaching staff and children; the data acquisition unit is used to collect video data and sound data of children during the learning process; the data storage unit is used to store the data acquired by the data acquisition unit; the data processing unit is used to pre-process the collected child data; the child attention monitoring unit is used to monitor the attention state of children during the learning process; the child attention distraction analysis unit is used to analyze the reasons when children are distracted; the feedback suggestion generation unit is used to generate teaching adjustment suggestion information; the prompt unit is used to remind teaching staff of the current child state; the interaction unit is used for teaching staff to conduct visual interaction with children.

7. A preschool education visual interactive system according to claim 6, characterized in that: The data processing unit includes: obtaining historical video data and historical sound data of multiple children in the data storage unit; the preprocessing process of the historical video data includes: dividing the historical video data according to video frames to obtain multiple first historical images; performing image denoising on multiple first historical images to obtain multiple second historical images; scaling multiple second historical images to a uniform scale to obtain multiple third historical images; performing image enhancement on multiple third historical images to obtain multiple fourth historical images; integrating multiple fourth historical images to obtain a historical image set; the preprocessing process of the historical sound data includes: dividing the historical sound data according to sound intervals to obtain multiple first historical sound data segments; performing sound denoising on multiple first historical sound data segments to obtain multiple second historical sound data segments; performing sound enhancement on multiple second historical sound data segments to obtain multiple third historical sound data segments; integrating multiple third historical sound data segments to obtain a historical sound set.

8. The preschool education visual interactive system according to claim 6, characterized in that: The child attention monitoring unit adopts a trained child state monitoring model for monitoring; wherein, the training process of the child state monitoring model includes: inputting data samples in the child state training set into the input layer according to the data type; wherein, the input layer includes: an image input submodule and a sound input submodule; inputting the image of the image input submodule into the image data processing layer to obtain the hand area image, the face area image and the eye area image in the image; inputting the sound data of the sound input submodule into the sound data processing layer to obtain text information and sound information; inputting the hand area image, the face area image and the eye area image into the image feature extraction layer to obtain a hand feature set, a face feature set and an eye feature set; inputting the text information into the text feature extraction layer to obtain a text feature set; inputting the sound information into the sound feature extraction layer to obtain a sound feature set; Input the hand feature set, the face feature set and the eye feature set to the sub-item area weighted layer to obtain the hand weighted feature set, the face weighted feature set and the eye weighted feature set; input the text feature set and the sound feature set to the merging layer to obtain the text-sound feature merged set; input the hand weighted feature set, the face weighted feature set, the eye weighted feature set and the text-sound feature merged set to the output layer to obtain the attention state monitoring result; wherein the output layer is composed of a classification model; calculate the monitoring error between the attention state monitoring result and the labeling result; optimize the child state monitoring model according to the monitoring error; when the monitoring error reaches convergence, the child state monitoring model training is completed.

9. The preschool education visual interactive system according to claim 6, characterized in that: The child attention distraction analysis unit uses an attention distraction analyzer to perform attention distraction analysis; wherein the attention distraction analyzer includes: According to the monitoring result of the child attention monitoring unit, the child with distracted attention is regarded as the target child; Obtaining the target child's concentration duration, basic information, and teaching content; wherein the basic information includes: age, gender, and personality description; Performing information processing on the concentration duration, the basic information, and the teaching content to obtain an influence vector; The influence vector is input into the attention distraction analysis function to obtain an analysis vector; wherein the calculation formula of the analysis vector is: ;in, is the analytical vector of the target child; The dispersion of the target children; is a distribution vector of influencing factors of the target child; is the attention distraction parsing function; is the focus duration vector; is the basic information feature vector; is the feature vector of teaching content.

10. The preschool education visual interactive system according to claim 6, characterized in that: The feedback suggestion generation unit uses a suggestion intelligent generation network to generate the teaching adjustment suggestion information, and the process includes: constructing a preschool education expert knowledge base based on the experience and knowledge of preschool education experts; wherein the content of the preschool education expert knowledge base includes: children's teaching cases, teaching response experience strategies and effect feedback; performing feature engineering on the content in the preschool education expert knowledge base to obtain a preschool education knowledge feature data set; evaluating each data in the preschool education knowledge feature data set to obtain multiple preschool education knowledge evaluation values; mapping the data in the preschool education knowledge feature data set with the multiple preschool education knowledge evaluation values ​​to construct a preschool education knowledge graph; obtaining personality description information of target children and non-target children; converting the personality description information into personality description features, and constructing a personality description feature set; obtaining a teaching adjustment suggestion text vector based on the analytic vector, the personality description feature set and the preschool education knowledge graph; and converting the teaching adjustment suggestion text vector into the teaching adjustment suggestion information.

Images