A control method of a teaching-learning machine and related equipment

Abstract

The application provides a teaching learning machine control method and related equipment, and relates to the technical field of learning machine technology. The teaching learning machine control method comprises the following steps: acquiring first learning state data, pushing corresponding learning content to an interactive interface; controlling a camera to start, acquiring facial image information of a student, determining a facial expression and an attention degree corresponding to the facial image information; weighting and fusing the facial expression and the attention degree to determine a learning confusion degree of the student to the learning content; generating teaching guidance information corresponding to the learning confusion degree, and adjusting a presentation mode or a difficulty level of the learning content according to the teaching guidance information; updating the first learning state data into second learning state data, pushing the second learning state data and the teaching guidance information to a teacher terminal, and continuously monitoring teacher feedback information of the teacher terminal; optimizing the teaching guidance information according to the teacher feedback information, and re-determining the presentation mode or the difficulty level of the learning content.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of learning machines, in particular to a control method of a teaching learning machine, a control device of a teaching learning machine and a teaching learning machine. BACKGROUND

[0002] As an intelligent device for assisting education, teaching learning machines have been widely used in family and classroom environments, aiming to provide personalized learning experience in a digital way. Existing teaching learning machines usually have a pre-installed teaching resource library, which can play videos, set questions and automatically correct them according to the subject and chapter selected by the user. The teaching logic is mostly based on linear, pre-set knowledge tree structure, for example, after the user completes the current knowledge point practice, the next knowledge point is unlocked.

[0003] However, such devices have three key technical defects: first, in terms of state perception, only surface data such as correct answer rate and time consumption can be obtained, and the real learning state of students cannot be accurately judged through multi-modal data fusion such as facial expression recognition and gaze tracking; second, in terms of strategy generation, the teaching adjustment is only based on simple rule matching, lacking quantitative assessment of learning confusion degree and dynamic strategy matrix support; finally, in terms of human-computer collaboration, the existing devices cannot realize real-time data synchronization and feedback loop of the teacher terminal, resulting in serious lag in teaching intervention. These defects make it difficult for existing devices to break through the limitations of mechanical indoctrination and provide precise teaching guidance for personalized needs of students. SUMMARY

[0004] The main purpose of the present application is to provide a control method of a teaching learning machine, aiming to improve the accuracy of student learning state perception and realize dynamic adjustment of teaching strategy.

[0005] To achieve the above purpose, the present application provides a control method of a teaching learning machine, the teaching learning machine comprising an interactive interface and a camera device, the control method of the teaching learning machine comprising:

[0006] obtaining current first learning state data of a student, pushing learning content corresponding to the first learning state data to the interactive interface, the first learning state data comprising learning progress and interactive feedback;

[0007] controlling the camera device to start, obtaining facial image information of the student, determining facial expression and attention degree corresponding to the facial image information;

[0008] weighting the facial expression and attention degree, determining the learning confusion degree of the student on the learning content;

[0009] generate teaching guidance information corresponding to the learning confusion degree, and adjust the presentation mode or difficulty level of the learning content according to the teaching guidance information;

[0010] update the first learning state data to second learning state data, push the second learning state data and the teaching guidance information to the teacher terminal, and continuously monitor the teacher feedback information of the teacher terminal;

[0011] optimize the teaching guidance information according to the teacher feedback information, and redetermine the presentation mode or difficulty level of the learning content.

[0012] Optionally, the control of the camera starts to obtain the face image information of the student, and determines the facial expression and attention degree corresponding to the face image information, comprising:

[0013] obtain the current ambient light parameter and the orientation information of the student relative to the camera;

[0014] adjust the shooting parameter of the camera based on the ambient light parameter and the orientation information of the student relative to the camera;

[0015] control the camera to obtain a plurality of face image information of the student according to the shooting parameter;

[0016] extract the key facial feature points of each face image information in sequence according to the shooting time sequence;

[0017] sort a plurality of key facial feature points to determine the motion trajectory and distribution state of the key facial feature points, and determine the facial expression corresponding to the motion trajectory and distribution state of the key facial feature points;

[0018] extract the position change trajectory of the eyeball pupil and the eyelid opening degree in a plurality of face image information, and determine the line of sight focus degree and the blink frequency;

[0019] weighting fusion of the line of sight focus degree and the blink frequency, to determine the attention degree, the attention degree includes full concentration, attention concentration, attention dispersion.

[0020] Optionally, the sorting of a plurality of key facial feature points to determine the motion trajectory and distribution state of the key facial feature points, and determining the facial expression corresponding to the motion trajectory and distribution state of the key facial feature points, comprises:

[0021] coordinate normalization processing is performed on the extracted plurality of key facial feature points;

[0022] calculate the coordinate change amount of the same key facial feature point in a continuous time sequence, and generate a motion vector of each key facial feature point.

[0023] The motion vectors of each key facial feature point are aggregated to determine a comprehensive motion trend of all key facial feature points, and a distribution state is generated;

[0024] A motion trajectory pattern representing the overall facial muscle motion is generated according to the motion vectors and the distribution state;

[0025] The motion trajectory pattern is matched with a preset typical expression template library, which stores standard motion trajectory patterns corresponding to four basic expressions of happiness, calmness, confusion and boredom;

[0026] The similarity between the motion trajectory and each standard motion trajectory pattern in the typical expression template library is calculated;

[0027] The expression corresponding to the standard motion trajectory pattern with the highest similarity is selected as the finally determined facial expression;

[0028] The position change trajectory of the eyeball pupil and the eyelid opening degree in the plurality of facial image information are extracted to determine the gaze focus degree and the blink frequency, including:

[0029] The eyeball region is located in each frame of facial image information, and the center coordinates of the pupil are identified;

[0030] The movement path of the pupil center coordinates in consecutive multiple frames of facial image information is tracked to form a position change trajectory;

[0031] The total length of the position change trajectory in a unit time is calculated, and if the length is lower than a first threshold, it is determined that the gaze focus degree is high, and if it is higher than a second threshold, it is determined that the gaze focus degree is low;

[0032] The vertical distance between the upper eyelid and the lower eyelid in each frame of image is extracted;

[0033] If it is monitored that in consecutive two frames of images, the eyelid opening degree changes from higher than the threshold to lower than the threshold and then recovers, it is counted as one blink event;

[0034] The total number of blink events occurring in a unit time window is counted to determine the blink frequency.

[0035] Optionally, the learning confusion degree of the student to the learning content is determined by weighting and fusing the facial expression and the attention degree, including:

[0036] The facial expression is assigned a first weight factor, and the attention degree is assigned a second weight factor, and the sum of the first weight factor and the second weight factor is one;

[0037] map the facial expression to an expression quantization value in a first preset numerical interval, and map the attention degree to an attention quantization value in a second preset numerical interval;

[0038] multiply the first weight factor and the expression quantization value to obtain a first weighted value, and multiply the second weight factor and the attention quantization value to obtain a second weighted value;

[0039] add the first weighted value and the second weighted value to generate a first fusion score value;

[0040] obtain an inherent difficulty coefficient of the current learning content;

[0041] determine the inherent difficulty coefficient as a gain coefficient, and gain the first fusion score value to generate a target fusion score value;

[0042] compare the target fusion score value with a plurality of difficulty threshold intervals one by one to determine the learning confusion degree.

[0043] Optionally, the teaching guidance information includes a strategy identifier.

[0044] The teaching guidance information corresponding to the learning confusion degree is generated, and the presentation mode or the difficulty level of the learning content is adjusted according to the teaching guidance information, including:

[0045] a multi-level teaching strategy matrix is preset, and the multi-level teaching strategy matrix takes the learning confusion degree level as the row and takes the knowledge point type as the column;

[0046] one or more candidate teaching guidance strategies are located in the multi-level teaching strategy matrix according to the learning confusion degree level and the knowledge point type to which the content being learned belongs;

[0047] If the strategy identifier indicates adjusting the presentation mode, a first execution parameter of the teaching guidance information is analyzed, and the interactive interface is controlled to convert the current pure text discussion into a structured mind map, or insert a demonstrative animation, or highlight a key formula;

[0048] If the strategy identifier indicates adjusting the difficulty level, a target difficulty value in the first execution parameter is used to filter out a replacement question set meeting the target difficulty value from a preset question bank, and the replacement question set is pushed to the student through the interactive interface.

[0049] Optionally, the one or more candidate teaching guidance strategies are located in the multi-level teaching strategy matrix according to the learning confusion degree level and the knowledge point type to which the content being learned belongs, including:

[0050] quantifying the learning confusion degree level into an integer value as a row index of the matrix;

[0051] encoding the knowledge point type into a category identifier as a column index of the matrix;

[0052] locating to an initial strategy unit in the multi-level teaching strategy matrix according to the row index and the column index, the initial strategy unit including a basic strategy set;

[0053] calling the historical strategy application effect record of the student, and calculating a historical average effective rate of each strategy in the basic strategy set;

[0054] attaching a time decay coefficient to the historical average effective rate of each strategy to determine a current timeliness weight value of each strategy;

[0055] sorting all strategies in the basic strategy set in descending order according to the current timeliness weight value, and screening top-ranked strategies as final candidate teaching guidance strategies.

[0056] Optionally, the method further includes:

[0057] encrypting the first learning state data, the learning confusion degree, the teaching guidance information, and the adjusted learning content segment after summarizing them, and encapsulating them into a second learning state data packet;

[0058] linking the second learning state data packet to a data packet content including a timestamp and a digital signature, and pushing the second learning state data packet to a corresponding teacher terminal through a secure communication link;

[0059] controlling the teacher terminal to analyze and visually display the data packet content, and generating and providing a teacher feedback interface including preset options and a custom input box;

[0060] starting one or more background monitoring services to continuously monitor whether a feedback data packet is returned from a network port of the teacher terminal in a polling manner;

[0061] decrypting and analyzing the received feedback data packet to extract a preset feedback instruction selected by the teacher, text annotation information input by the teacher, or a supplementary teaching resource file uploaded by the teacher, and generating the teacher feedback information.

[0062] Optionally, the control method of the teaching learning machine further includes:

[0063] If the teacher feedback information of the teacher terminal is not monitored within the preset time length, a local emergency guidance mode is started;

[0064] In the case of the local emergency guidance mode, historical learning behavior data of the student is acquired;

[0065] A preset teaching problem solution library is called through the interactive interface;

[0066] According to the historical learning behavior data, an adaptive auxiliary teaching scheme is matched in the teaching problem solution library based on an optimal strategy;

[0067] The teaching guidance information is modulated based on the auxiliary teaching scheme, and the presentation mode or the difficulty level of the learning content is re-determined.

[0068] In addition, to achieve the above-mentioned purpose, the present application also provides a control device, which comprises a memory, a processor and a control program of a teaching learning machine stored on the memory and executable on the processor, and the control program of the teaching learning machine is configured to implement the control method of the teaching learning machine as described above.

[0069] In addition, to achieve the above-mentioned purpose, the present application also provides a teaching learning machine comprising the control device as described above.

[0070] The teaching learning machine of the embodiment of the present application comprises an interactive interface and a camera device, and the control method of the teaching learning machine comprises the following steps: acquiring current first learning state data of a student, and pushing learning content corresponding to the first learning state data to the interactive interface, wherein the first learning state data comprises learning progress and interactive feedback; then controlling the camera device to start, acquiring facial image information of the student to determine facial expression and attention degree corresponding to the facial image information; then weighting and fusing the facial expression and the attention degree to determine learning confusion degree of the student to the learning content; generating teaching guidance information corresponding to the learning confusion degree, and adjusting the presentation mode or the difficulty level of the learning content according to the teaching guidance information; updating the first learning state data to second learning state data, pushing the second learning state data and the teaching guidance information to a teacher terminal, and continuously monitoring teacher feedback information of the teacher terminal; and finally optimizing the teaching guidance information according to the teacher feedback information, and re-determining the presentation mode or the difficulty level of the learning content. In this way, by acquiring the learning state data and dynamically adjusting the teaching strategy in combination with the facial expression and the attention degree, and realizing real-time feedback of the teacher terminal, the learning state perception accuracy of the student can be improved, and the teaching strategy can be dynamically adjusted. BRIEF DESCRIPTION OF DRAWINGS

[0071] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0072] The technical solutions in the embodiments of the present application or the prior art will be described below with reference to the drawings needed in the embodiments or prior art description. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0073] Figure 1 Flowchart of the control method of the teaching learning machine according to an embodiment of the present application;

[0074] Figure 2 Flowchart of the control method of the teaching learning machine according to another embodiment of the present application;

[0075] Figure 3 Flowchart of step S250 in the method shown in Figure 2 Flowchart of step S260 in the method shown in

[0076] Figure 4 Flowchart of step S260 in the method shown in Figure 2 Flowchart of step S260 in the method shown in

[0077] Figure 5 Flowchart of the control method of the teaching learning machine according to another embodiment of the present application;

[0078] Figure 6 Flowchart of the control method of the teaching learning machine according to another embodiment of the present application;

[0079] Figure 7 Flowchart of the control method of the teaching learning machine according to another embodiment of the present application;

[0080] Figure 8 Flowchart of the control method of the teaching learning machine according to another embodiment of the present application;

[0081] Figure 9 Flowchart of the control method of the teaching learning machine according to another embodiment of the present application.

[0082] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the drawings. DETAILED DESCRIPTION

[0083] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below, obviously, the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The well-known modules, units and their connections, links, communications or operations between each other are not shown or not described in detail. And the described features, architectures or functions can be combined in any way in one or more embodiments. Those skilled in the art should understand that the following various embodiments are only used for illustration, and not used to limit the protection scope of the present application. It can also be easily understood that the modules or units or processing methods in each embodiment described herein and shown in the drawings can be combined and designed in various different configurations. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0084] In the following embodiments, the definition of various nouns or methods is generally based on the broad concept that can be implemented on the premise of the disclosed content in the embodiments, except in cases where it is logically impossible. Under such understanding, various specific sub-limits of the nouns or methods should be considered as the invention content of the present application, and should not be interpreted in a narrow sense or biased interpretation because the specific limit is not disclosed in the specification. Similarly, the order of steps in the method is flexible and variable on the premise that it can be logically implemented. The specific sub-limits of the broad concept of various nouns or methods are within the scope of protection of the present application.

[0085] As an intelligent device for assisting education, the teaching and learning machine has been widely used in family and classroom environments, aiming to provide personalized learning experience in a digital way. The existing teaching and learning machine usually has a pre-installed teaching resource library, which can play videos, set questions and automatically correct them according to the subject and chapter selected by the user. Its teaching logic is mostly based on linear, pre-set knowledge tree structure, for example, after the user completes the current knowledge point practice, the next knowledge point is unlocked.

[0086] However, the core defect of such devices is their "passivity" and "staticity". They cannot actively and accurately identify the specific and deep teaching problems encountered by students in the learning process (such as misunderstanding of a core concept, specific thinking barriers or understanding discontinuity caused by inattention). The adjustment of their teaching strategies often depends on simple answer correctness or time consumption, lacking multi-dimensional perception and depth analysis of the causes of the problem. Therefore, when students are in learning difficulties, the existing devices are difficult to provide truly targeted adaptive teaching feedback, and the adjustment of their teaching content has serious lag and blindness, and essentially still cannot get rid of the "mechanized indoctrination" mode, and the teaching efficiency and effect are limited.

[0087] A core technical problem to be solved in the prior art is how to enable the teaching learning machine to actively and intelligently identify and analyze the teaching difficulties encountered by students, and accordingly realize dynamic, accurate and automatic adjustment of teaching content and strategy, so as to break through the limitations of the traditional static teaching mode.

[0088] The main solution of the embodiment of the present application is: obtaining the first learning state data of the student, and pushing the learning content corresponding to the first learning state data to the interactive interface, wherein the first learning state data includes learning progress and interactive feedback; then controlling the camera to start and obtaining the facial image information of the student to determine the facial expression and attention degree corresponding to the facial image information; then weighting and fusing the facial expression and attention degree to determine the learning confusion degree of the student to the learning content; and generating teaching guidance information corresponding to the learning confusion degree, adjusting the presentation mode or difficulty level of the learning content according to the teaching guidance information; then updating the first learning state data to the second learning state data, pushing the second learning state data and the teaching guidance information to the teacher terminal, and continuously monitoring the teacher feedback information of the teacher terminal; finally, optimizing the teaching guidance information according to the teacher feedback information, and re-determining the presentation mode or difficulty level of the learning content.

[0089] In the embodiment, the following is described with the control device as the execution subject for the convenience of description.

[0090] The present application provides a solution, which dynamically adjusts the teaching strategy by obtaining learning state data and combining facial expression and attention degree, and realizes real-time feedback of the teacher terminal, which can improve the accuracy of student learning state perception and realize dynamic adjustment of teaching strategy.

[0091] Therefore, the present application provides a control method of a teaching learning machine; it can be understood that the control device for storing and executing the following method is arranged in the teaching learning machine, and the control device can be realized by a main controller, such as MCU (Micro controller Unit), DSP (Digital Signal Process), FPGA (Field Programmable Gate Array), SOC (System On Chip) and the like.

[0092] In the prior art, a teaching learning machine usually operates based on a preset resource library and a linear knowledge structure, and adjusts teaching strategies depending on surface data such as correct answer rate or time consumption. Such a teaching learning machine cannot actively identify deep-seated problems encountered by students in the learning process, such as misunderstanding of core concepts or understanding fault caused by distraction. For example, when a student is confused by the logical disorder of formula derivation in a mathematics class, the existing teaching learning machine can only repeat the same question type according to the wrong answer, and cannot capture facial expression changes or visual line wandering states, resulting in delayed and non-targeted teaching feedback.

[0093] That is, the traditional method only relies on single-dimensional data, resulting in a high misjudgment rate. In order to solve the above problems, a multi-modal perception mechanism needs to be constructed. By analyzing the dynamic changes of facial expressions and attention states of students, combined with learning progress data, a more accurate confusion recognition model can be established. By adjusting the presentation method of teaching content in real time, it is more effective to solve understanding obstacles than simply modifying the difficulty of the question. Therefore, the technical idea is formed: synchronously collecting learning state data and biological feature data, generating a confusion degree index through weighted fusion, and realizing dynamic adjustment through parallel teaching strategy library, and introducing a teacher feedback mechanism to optimize the algorithm model.

[0094] Based on the above content, refer to Figure 1 In an embodiment of the present application, the teaching learning machine includes an interactive interface and a camera device, and the control method of the teaching learning machine includes steps S100-S600, wherein:

[0095] S100, acquiring current first learning state data of a student, pushing learning content corresponding to the first learning state data to the interactive interface, the first learning state data including learning progress and interactive feedback;

[0096] S200, controlling the camera device to start, acquiring facial image information of the student, and determining the facial expression and attention degree corresponding to the facial image information;

[0097] S300, weighted fusion of the facial expression and attention degree, determination of the learning confusion degree of the student to the learning content;

[0098] S400, generating teaching guidance information corresponding to the learning confusion degree, and adjusting the presentation method or difficulty level of the learning content according to the teaching guidance information;

[0099] S500, updating the first learning state data to second learning state data, pushing the second learning state data and the teaching guidance information to a teacher terminal, and continuously monitoring teacher feedback information of the teacher terminal;

[0100] S600, optimizing the teaching guidance information according to the teacher feedback information, re-determining the presentation mode or difficulty level of the learning content.

[0101] The learning state data is a data set reflecting the knowledge mastery and interactive behavior of the student, which can be realized by recording the completion rate of practice questions, chapter learning time length, and touch screen operation frequency in the answering process, and is used for dynamically matching the teaching content. The facial image information is a facial region video stream captured by a camera device, which can use an infrared light supplement camera to collect high-definition images in a low-light environment, and is used for extracting eye and mouth muscle movement features. The weighted fusion is a normalization processing of multi-dimensional data and a superposition according to a preset weight, which can use a linear weighting algorithm to combine the expression confidence and the attention score, and eliminate the noise interference of a single data source. The teaching guidance information is a strategy instruction driving content adjustment, which can call a mind map generation module or a question difficulty filter based on a strategy identifier, to realize the presentation mode conversion.

[0102] The control device first retrieves the current learning progress of the student from the local database, for example, 30 minutes of learning in the trigonometric function chapter, and generates interactive feedback indicators in combination with the error rates of the last three practice questions. The camera device is automatically started when it detects that the student looks up and gazes at the screen, calculates the line of sight focus degree through a pupil tracking algorithm, and analyzes the inter-brow wrinkle depth and the mouth corner droop angle to determine the confusion expression. When the expression weight is 0.6 and the attention weight is 0.4, the control device determines that the confusion degree reaches the second threshold value, and triggers the teaching content adjustment instruction. At this time, the interactive interface converts the pure text analytical expression into a three-dimensional geometric animation demonstration, and synchronously encrypts and transmits the adjusted learning data to the teacher terminal. The teacher confirms the validity of the strategy by checking the preset options, and the control device updates the weight coefficients accordingly to optimize the fusion calculation accuracy next time.

[0103] Compared with the prior art, the embodiment solves the limitation of the traditional method relying on single answering data by constructing a dynamic evaluation model through multi-source data fusion. The prior art can only passively adjust the questions after the answering error, while the embodiment can intervene in advance through biometric recognition during the answering process. For example, when explaining the problem of physical mechanics, the traditional device provides a hint only after the student submits an incorrect answer, while the embodiment inserts a force analysis animation as soon as the student's frown frequency suddenly increases and the line of sight deviates, advancing the intervention time by 2-3 minutes.

[0104] Through the above technical solutions, the embodiment realizes real-time dynamic adjustment of the teaching strategy, effectively shortening the time difference between problem identification and intervention response. The control device can accurately distinguish between true confusion caused by excessively difficult knowledge points and false confusion caused by temporary distraction, avoiding unnecessary teaching interruptions. The teacher terminal feedback mechanism ensures continuous optimization of the algorithm model, gradually improving the specificity and sensitivity of the confusion degree determination.

[0105] The teaching learning machine of the embodiment comprises an interactive interface and a camera device, and the control method of the teaching learning machine comprises the following steps: acquiring current first learning state data of a student, and pushing learning content corresponding to the first learning state data to the interactive interface, wherein the first learning state data comprises learning progress and interactive feedback; then controlling the camera device to start, acquiring facial image information of the student to determine facial expression and attention degree corresponding to the facial image information; then weighting and fusing the facial expression and the attention degree to determine learning confusion degree of the student to the learning content; generating teaching guidance information corresponding to the learning confusion degree, adjusting presentation mode or difficulty level of the learning content according to the teaching guidance information; updating the first learning state data to second learning state data, pushing the second learning state data and the teaching guidance information to a teacher terminal, and continuously monitoring teacher feedback information of the teacher terminal; and finally optimizing the teaching guidance information according to the teacher feedback information, and re-determining the presentation mode or the difficulty level of the learning content. In this way, the learning state data is acquired, the teaching strategy is dynamically adjusted according to the facial expression and the attention degree, and the real-time feedback of the teacher terminal is realized, so that the learning state perception accuracy of the student is improved, and the teaching strategy is dynamically adjusted.

[0106] Optionally, with reference to Figure 2 , another embodiment of the present application provides a control method of a teaching learning machine, based on the above Figure 1 mentioned embodiments, the camera device is controlled to start, facial image information of the student is acquired, and facial expression and attention degree corresponding to the facial image information are determined, comprising steps S210-S270, wherein:

[0107] S210, acquiring current environmental light parameter and orientation information of the student relative to the camera device;

[0108] S220, adjusting shooting parameters of the camera device based on the environmental light parameter and the orientation information of the student relative to the camera device;

[0109] S230, controlling the camera device to acquire multiple facial image information of the student according to the shooting parameters;

[0110] S240, extracting key facial feature points of each facial image information one by one in shooting time sequence;

[0111] S250, sorting multiple key facial feature points to determine motion trajectory and distribution state of the key facial feature points, and determining facial expression corresponding to the motion trajectory and distribution state of the key facial feature points;

[0112] S260, extract the position change trajectory of the eyeball pupil and the eyelid opening degree in the plurality of face image information, and determine the line of sight focusing degree and the blink frequency;

[0113] S270, weighted fusion of the line of sight focusing degree and the blink frequency, determine the attention degree, the attention degree includes full concentration, attention concentration, attention dispersion.

[0114] Wherein, the ambient light parameter refers to the ambient illuminance value measured by the light sensor, which can adopt the light intensity detection module to collect the ambient light data in real time, and the ambient light parameter is used for dynamically adjusting the exposure compensation and white balance parameter of the camera device. The azimuth information refers to the relative position relationship between the student's head and the camera device, which can obtain three-dimensional coordinate data through infrared distance sensor or binocular vision positioning technology. The key facial feature points refer to the dynamic marker points generated when the facial muscles move, which can adopt the face key point detection algorithm to identify 64 feature coordinate points in the eyebrow, mouth corner, nose wing and other regions. The motion trajectory refers to the displacement path of the feature points in the time dimension, which can track the continuous movement process of the feature points through the optical flow method or Kalman filtering algorithm. The eyelid opening degree refers to the vertical distance between the upper and lower eyelids, which can calculate the pixel distance after extracting the eyelid edge coordinates through image segmentation technology.

[0115] Wherein, in the implementation process, first, the light intensity and angle distribution of the learning scene are monitored in real time through the environment sensor, combined with the spatial coordinate data of the student's seat, the ISO sensitivity, aperture size and light supplement intensity of the camera device are automatically optimized to ensure the clarity of the face image acquisition. Subsequently, a plurality of face images are continuously captured at a rate of 30 frames per second, and the dynamic feature point coordinates of the eyebrow, mouth corner and other regions in each frame of image are extracted through the convolutional neural network. After arranging these coordinates in time sequence, the moving direction and speed of the feature points are analyzed by using the trajectory clustering algorithm, and the comprehensive motion trend spectrum representing the facial muscle activity is generated. In addition, the pupil tracking algorithm is used to record the eyeball rotation angle and frequency, and the blink frequency per unit time is calculated by combining the eyelid spacing change data, and finally the attention level is calculated by weighting the line of sight stability and the blink frequency.

[0116] The existing teaching equipment only relies on a single camera with fixed angle to collect static images, which cannot adapt to the image blur problem caused by light change or student movement. The present embodiment can accurately capture subtle facial dynamic changes in complex environmental conditions through dynamic adjustment of camera parameters and multi-dimensional feature analysis. The attention judgment method based on the time-consuming of answering in the prior art has serious lag, while the present embodiment realizes the millisecond-level dynamic evaluation of the attention state by tracking the pupil trajectory and eyelid movement in real time.

[0117] The embodiment effectively solves the face feature recognition error problem caused by environmental interference of the traditional device, and improves the accuracy of expression and attention analysis. By fusing the dynamic feature point track in the multiple image frames and the eye physiological data, different learning states such as student confusion and distraction can be accurately distinguished, and reliable data support is provided for real-time adjustment of subsequent teaching strategies.

[0118] Optionally, with reference to Figure 3 and 4 , a control method of a teaching and learning machine is further provided in another embodiment of the present application. Based on the above Figure 2 indicated embodiment, the motion track and distribution state of the key facial feature points are determined by sorting a plurality of key facial feature points, and the facial expression corresponding to the motion track and distribution state of the key facial feature points is determined, including steps S251-S257, wherein:

[0119] S251, coordinate normalization processing is performed on the plurality of key facial feature points extracted, so as to eliminate the scale difference caused by the student head movement distance;

[0120] S252, the coordinate change amount of the same key facial feature point on the continuous time sequence is calculated, and the motion vector of each key facial feature point is generated;

[0121] S253, the motion vector of each key facial feature point is summarized to determine the comprehensive motion trend of all key facial feature points, and the distribution state is generated;

[0122] S254, the motion track mode representing the overall facial muscle motion is generated according to the motion vector and the distribution state;

[0123] S255, the motion track mode is matched with a preset typical expression template library, and the typical expression template library stores standard motion track modes corresponding to four basic expressions of happiness, calmness, confusion and boredom;

[0124] S256, the similarity between the motion track and each standard motion track mode in the typical expression template library is calculated;

[0125] S257, the expression corresponding to the standard motion track mode with the highest similarity is selected as the finally determined facial expression.

[0126] The position change track of the eyeball pupil and the eyelid opening degree in the plurality of facial image information are extracted, the line of sight focusing degree and the blinking frequency are determined, and the method comprises steps S261-S266, wherein:

[0127] S261, the eyeball region is located in each frame of facial image information, and the center coordinates of the pupil are identified;

[0128] S262, tracking the moving path of the pupil center coordinate in the continuous multi-frame face image information to form a position change trajectory;

[0129] S263, calculating the total length of the position change trajectory in a unit time, if the length is lower than a first threshold, determining that the visual line focusing degree is high, if the length is higher than a second threshold, determining that the visual line focusing degree is low;

[0130] S264, extracting the vertical distance between the upper eyelid and the lower eyelid in each frame of image, the vertical distance between the upper eyelid and the lower eyelid being the eyelid opening degree;

[0131] S265, if the eyelid opening degree is monitored to change from being higher than a threshold to being lower than a threshold and then recovering in the continuous two frames of image, counting as one blink event;

[0132] S266, counting the total number of blink events occurring in a unit time window to determine the blink frequency.

[0133] Wherein, the coordinate normalization processing refers to converting the face feature point coordinates at different distances into relative coordinates at a unified scale, which can be realized by affine transformation or proportional scaling algorithm, and the coordinate normalization processing is used to eliminate the image scale difference caused by the forward leaning or backward leaning of the student's body. The motion vector refers to the displacement vector of the same feature point at the same time point, which can be obtained by calculating the difference between adjacent frame coordinates, and the motion vector is used to quantify the instantaneous dynamic change of the facial muscle. The comprehensive motion trend refers to the statistical distribution characteristics of the motion vectors of multiple feature points, and the main component analysis method can be used to extract the main motion direction, and the comprehensive motion trend is used to represent the continuity change of the overall expression. The standard motion trajectory mode refers to the typical expression dynamic feature template established by machine learning in advance, which can be matched by using the dynamic time warping algorithm, and the standard motion trajectory mode is used to classify the real-time collected facial motion data into known expression categories. The position change trajectory refers to the moving path of the eyeball pupil in the continuous image, which can be smoothed by Kalman filtering algorithm, and the position change trajectory is used to analyze the coherence of the visual line movement. The eyelid opening degree refers to the vertical distance between the upper and lower eyelids, which can be calculated by the Euclidean distance after extracting the eyelid contour by edge detection algorithm, and the eyelid opening degree is used to quantify the amplitude characteristics of the blink action.

[0134] Wherein, after acquiring the student facial image, first, the key feature points are normalized to eliminate the head displacement interference, and then the subtle dynamic changes of the facial muscles are captured by analyzing the feature point motion vectors. The overall facial motion trajectory is generated by summarizing the motion trends of the feature points, and is compared with the preset four basic expression templates in similarity, so that the expressions closely related to the learning state such as confusion and boredom are accurately recognized. In addition, whether the line of sight is stably focused is judged by tracking the length of the pupil movement path, and the blinking frequency is combined with the blinking frequency to establish a quantitative evaluation index of the attention degree. For example, when the pupil movement trajectory length is detected to be lower than the threshold value and the blinking frequency is within the normal range, it can be determined that the student is in an attention concentrated state; if the facial motion trajectory matches the confusion expression template with a matching degree exceeding a set threshold value, a learning confusion warning is triggered.

[0135] The control method of the existing teaching learning machine only recognizes the basic expressions through static images, cannot capture the dynamic change characteristics of the expressions, and does not consider the influence of head movement on feature extraction. The existing line of sight tracking technology mostly relies on special eye movement instrument equipment, and does not fully utilize the eye dynamic information collected by ordinary cameras. The embodiment realizes high-precision expression recognition and attention monitoring under ordinary hardware conditions through dynamic trajectory analysis and multi-frame feature fusion.

[0136] The embodiment can effectively distinguish the attention fluctuation caused by temporary distraction from the persistent confusion expression caused by knowledge understanding obstacles, accurately recognize the micro-expression changes caused by thinking jam in the learning process of students, and provide reliable biological behavior basis for subsequent teaching strategy adjustment by combining dynamic expression characteristics and line of sight focusing data, so that the error teaching intervention caused by single-dimensional data misjudgment is avoided.

[0137] Optionally, with reference to Figure 5 , a control method of a teaching learning machine is provided in another embodiment of the present application, based on the embodiments shown in Figure 3 and Figure 4 , the facial expression and the attention degree are weighted and fused to determine the learning confusion degree of the student to the learning content, including steps S310-S370, wherein:

[0138] S310, a first weight factor is assigned to the facial expression, and a second weight factor is assigned to the attention degree, and the sum of the first weight factor and the second weight factor is one;

[0139] S320, the facial expression is mapped to an expression quantitative value in a first preset numerical interval, and the attention degree is mapped to an attention quantitative value in a second preset numerical interval;

[0140] S330, multiplying the first weight factor and the expression quantization value to obtain a first weighted value, and multiplying the second weight factor and the attention quantization value to obtain a second weighted value;

[0141] S340, adding the first weighted value and the second weighted value to generate a first fusion score value;

[0142] S350, obtaining an inherent difficulty coefficient of the current learning content;

[0143] S360, determining the inherent difficulty coefficient as a gain coefficient, and gain processing the first fusion score value to generate a target fusion score value;

[0144] S370, comparing the target fusion score value with a plurality of preset difficulty threshold intervals one by one to determine the learning confusion degree.

[0145] The first weight factor and the second weight factor are coefficients for adjusting the influence of facial expressions and attention degree on the learning confusion degree, which can be dynamically adjusted by an optimization algorithm based on historical data analysis. For example, the attention degree weight is set to be higher in mathematical logic knowledge points, and the facial expression weight is increased in language understanding knowledge points. The expression quantization value is a mapping rule for converting expression types into numerical values, which can be assigned a discrete value of 0.8 for confused expressions and 0.3 for calm expressions. The inherent difficulty coefficient is a parameter reflecting the complexity of knowledge points, which can be generated by teaching outline annotation or exercise accuracy rate statistics, for example, setting the basic algebra knowledge point to 0.5 and the high-order geometry knowledge point to 0.9. The gain coefficient is an adjustment factor for amplifying or reducing the fusion score, which can be associated with learning content difficulty and learning state data by linear multiplication operation.

[0146] When the student appears confused expressions and attention is distracted, the expression quantization value can reach 0.8, and the attention quantization value decreases to 0.2. If the first weight factor is set to 0.6 and the second weight factor is set to 0.4, the first weighted value is 0.48, the second weighted value is 0.08, and the first fusion score value is 0.56 after addition. When the inherent difficulty coefficient of the learning content is 0.7, the target fusion score value is 0.392 generated by multiplication gain. Comparing this value with the preset threshold interval, for example, 0.3-0.4 corresponds to moderate confusion, it is determined that the student is currently in a learning state that needs intervention. In this process, the difficulty coefficient as the gain coefficient properly suppresses the low score confusion signal under simple knowledge points, while the same behavior data under high difficulty knowledge points is amplified to avoid misjudgment.

[0147] Compared with the prior art, the traditional method only relies on a single dimension data (for example, the correct rate of answering questions) to judge the learning state, and is easy to ignore the comprehensive influence of expression changes and attention fluctuations on understanding ability. The embodiment realizes the fusion analysis of multi-modal data through dynamic weight distribution, and calibrates the evaluation result in combination with the inherent difficulty of knowledge points, so that the judgment of the learning confusion degree considers not only the real-time behavior characteristics, but also the objective complexity of teaching content, and the recognition accuracy is significantly improved.

[0148] The embodiment can accurately distinguish between real confusion caused by too high difficulty of knowledge points and attention decline caused by temporary distraction, and avoid false triggering of teaching strategies. For example, when explaining a high-difficulty physical formula, even if the student temporarily shifts his gaze, the control device can still maintain a high confusion evaluation value through the inherent difficulty coefficient, so as to trigger the targeted formula disassembly animation push, realize the accurate matching of teaching intervention and knowledge difficulty.

[0149] Optionally, with reference to Figure 6 The present application also provides a control method of a teaching and learning machine, based on the above Figure 1 The embodiment shows that the teaching guidance information includes a strategy identifier; the generation of the teaching guidance information corresponding to the learning confusion degree, and the adjustment of the presentation mode or the difficulty level of the learning content according to the teaching guidance information, includes steps S410-S440, wherein:

[0150] S410, preset a multi-level teaching strategy matrix, the multi-level teaching strategy matrix takes the learning confusion degree level as the row and takes the knowledge point type as the column;

[0151] S420, according to the learning confusion degree level and the knowledge point type to which the learning content belongs, locate one or more candidate teaching guidance strategies in the multi-level teaching strategy matrix;

[0152] S430, if the strategy identifier indicates adjustment of the presentation mode, analyze the first execution parameter of the teaching guidance information, control the interactive interface to convert the current pure text discussion into a structured mind map, or insert a demonstration animation, or highlight the key formula;

[0153] S440, if the strategy identifier indicates adjustment of the difficulty level, according to the target difficulty value in the first execution parameter, filter out a set of alternative questions that meet the target difficulty value from a preset question bank, and push the set of alternative questions to the student through the interactive interface.

[0154] Wherein, the policy identifier refers to an identification code for distinguishing the teaching guidance policy type, which can be implemented by binary flag or string encoding, for example, "MODE" represents adjusting the presentation mode, and "LEVEL" represents adjusting the difficulty level. The multi-level teaching policy matrix refers to a policy database arranged in two dimensions according to the learning confusion degree and the knowledge point type, which can store the policy set in the form of hash table or two-dimensional array structure, for example, the confusion degree is divided into three levels of low, medium and high, and the knowledge point type is coded according to the subject chapter. The candidate teaching guidance policy refers to the operation instruction set selected from the policy matrix and adapted to the current learning situation, which can be dynamically adjusted in the order of policy recommendation by using the priority sorting algorithm combined with the historical application effect data.

[0155] Wherein, when the learning confusion degree level and the knowledge point type are determined, the control device locates the initial policy unit through the row and column indexes of the matrix, retrieves the historical policy application records of the student to calculate the efficiency of each policy, sorts the candidate policy list after adding the time decay coefficient. If the policy identifier is to adjust the presentation mode, the control device analyzes the execution parameters to trigger the content conversion of the interactive interface, for example, converting the text analysis of a mathematical proof question into a step-by-step animation demonstration. If the identifier is to adjust the difficulty level, the control device matches the question parameters in the question bank according to the target difficulty value, for example, selects the same type of question containing more graphical hints to replace the original question set.

[0156] The existing device only mechanically switches the preset content according to the correct answer rate, and cannot dynamically match the policy type based on the multi-dimensional learning state. For example, the traditional teaching machine only repeatedly pushes the same type of questions when detecting an increase in the error rate, while the embodiment can automatically select content reconstruction or difficulty reduction according to the confusion causes, and optimize the order of policy recommendation based on historical data. The embodiment solves the problem of single policy adjustment of existing teaching devices, and realizes precise adaptation of teaching content presentation mode and difficulty level. For example, for students who are confused due to complex logical chains, the control device automatically converts text analysis into a mind map to reduce cognitive load; for students who have difficulty understanding due to weak foundation, the control device dynamically replaces it with low-difficulty exercises with detailed steps, thereby improving the efficiency of personalized teaching.

[0157] Optionally, referring to Figure 7 , another embodiment of the present application provides a control method of a teaching machine, based on the above Figure 6 embodiment, one or more candidate teaching guidance policies are located in the multi-level teaching policy matrix according to the learning confusion degree level and the knowledge point type to which the content being learned belongs, including steps S421-S426, wherein:

[0158] S421, quantize the learning confusion degree level into an integer value as the row index of the matrix;

[0159] S422, encode the knowledge point type as a category identifier as a column index of the matrix;

[0160] S423, locate an initial strategy unit in the multi-level teaching strategy matrix according to the row index and the column index, the initial strategy unit including a basic strategy set;

[0161] S424, call the historical strategy application effect record of the student, and calculate the historical average effective rate of each strategy in the basic strategy set;

[0162] S425, attach a time decay coefficient to the historical average effective rate of each strategy to determine a current timeliness weight value of each strategy;

[0163] S426, sort all strategies in the basic strategy set in descending order according to the current timeliness weight value, and select the top several strategies as the final candidate teaching guidance strategies.

[0164] Wherein, quantifying the learning confusion degree level into an integer value means converting the understanding obstacle degree of the student to the learning content into a numerical form, which can be mapped by using a preset difficulty degree threshold interval, for example, the confusion degree is divided into an integer level of 1 to 5. The learning confusion degree level quantification method is convenient for quick index matching in the matrix. The knowledge point type is encoded as a category identifier, which means that different disciplines or knowledge modules are classified and marked. The encoding rule of letters and numbers can be used, for example, the algebra module in mathematics is encoded as MA01, and the geometry module is encoded as MG02. The knowledge point type encoding method is convenient for establishing the column dimension structure of the matrix. The multi-level teaching strategy matrix means a data structure for storing a set of teaching strategies corresponding to different confusion degrees and knowledge point types. It can be implemented by using a two-dimensional array or a database table. Each unit in the matrix stores a group of basic strategies. This structure supports quick positioning of the initial strategy set. The historical strategy application effect record means the learning effect data of the student after using different strategies in the past. It can be counted by the correct answer rate, interaction time or confusion degree change rate. The record is used to evaluate the actual effectiveness of the strategy. The time decay coefficient means a parameter for adjusting the weight value of the strategy according to the time of application. It can be calculated by using an exponential decay function, for example, the strategy applied in the last week is given a higher weight. The time decay coefficient is used to reflect the timeliness difference of the strategy.

[0165] When the control device detects that the student has confusion about the current learning content, the confusion degree is first converted into an integer value, for example, confusion level 3 corresponds to row index 3, the knowledge point type is identified and coded as a category identifier, for example, the mechanics module in physics is coded as PC03. According to the row index 3 and the column index PC03, the corresponding initial strategy unit is extracted from the matrix, which contains multiple basic strategies, such as increasing example explanation, inserting animation demonstration or splitting knowledge point steps. Subsequently, the control device calls the basic strategy data applied by the student in the past three months, and calculates the historical average efficiency of each strategy, for example, strategy A improves the correct rate by 15% in the past application, and strategy B improves the interaction time by 20%. A time decay coefficient is added to the historical data of each strategy, for example, the data one month ago has a weight of 0.8, the data two weeks ago has a weight of 0.9, and the current week data has a weight of 1.0, so as to calculate the current time weight value of the strategy. Finally, the control device sorts the strategies according to the weight value, and selects the top two strategies as candidate teaching guidance strategies, for example, strategy B and strategy A are preferentially recommended.

[0166] The traditional teaching device only matches fixed strategies according to preset rules, and cannot dynamically combine student historical data and time effectiveness factors. The embodiment establishes a multi-level matrix structure, accurately locates in combination with real-time confusion degree and knowledge point type, and dynamically optimizes strategy sorting based on historical application effect and time decay mechanism, so that the candidate strategy is more suitable for the current actual learning state of the student.

[0167] The embodiment can effectively solve the problem of rigid strategy matching of existing teaching devices, dynamically screen candidate strategies with high time effectiveness weight value, improve the adaptability and response speed of teaching guidance strategies, avoid the decline of learning efficiency caused by strategy lag, and realize fast retrieval and update of strategies through matrix structure and coding mechanism.

[0168] Optionally, with reference to Figure 8 , the present application further provides a control method of a teaching learning machine, based on the above Figure 1 indicated embodiment, updating the first learning state data to second learning state data, pushing the second learning state data and the teaching guidance information to a teacher terminal, and continuously monitoring the teacher feedback information of the teacher terminal, including steps S510-S550, wherein:

[0169] S510, the first learning state data, the learning confusion degree, the teaching guidance information and the adjusted learning content segment are integrated and encrypted, and packaged into a second learning state data package;

[0170] S520, link the data package content containing the timestamp and digital signature to the second learning state data package, and push it to the corresponding teacher terminal through a secure communication link.

[0171] S530, control the teacher terminal side analysis and visualization display the data packet content, generate and provide a teacher feedback interface containing preset options and custom input box;

[0172] S540, start one or more background monitoring services to continuously monitor the network port of the teacher terminal whether there is feedback data packet returned in a polling manner;

[0173] S550, decrypt and parse the received feedback data packet, extract the preset feedback instruction selected by the teacher, the text annotation information entered by the teacher or the supplementary teaching resource file uploaded by the teacher, and generate the teacher feedback information.

[0174] Wherein, encryption means converting data through cryptographic algorithms so that it cannot be read by unauthorized parties during transmission, AES symmetric encryption algorithm or RSA asymmetric encryption algorithm can be used to achieve it, to ensure the security of student privacy data and teaching information. Timestamp means the exact time mark of the data packet generation time, which can be realized by international standard time format (UTC) combined with millisecond level accuracy, and the timestamp is used to verify the timeliness of the data and prevent replay attacks. Digital signature means identity authentication of data packet using asymmetric encryption technology, which can be realized by generating data digest using hash function and then encrypting with private key, and digital signature is used to verify data integrity and authenticity. Secure communication link means encrypted network channel based on TLS protocol, which can be realized by configuring SSL certificate and key exchange protocol, to prevent data tampering during transmission. Polling means sending query request to target port periodically to detect feedback data, which can be realized by timer triggering HTTP GET request or Socket connection state check, to ensure real-time capture of feedback information.

[0175] Wherein, after the learning state of the teaching learning machine is updated, the key data including learning progress, confusion degree and adjustment strategy are encrypted and packaged, and the timestamp and digital signature are added to form a complete data packet. The data packet is transmitted to the teacher terminal through a secure link, and the terminal is analyzed and displayed in the form of charts, text summaries, etc. The teacher selects the preset instruction, inputs the text annotation or uploads the supplementary resources through the interface, and the control device monitors the feedback data in real time through the polling mechanism. The feedback data is decrypted to extract the effective information, and the feedback data is used to optimize the subsequent teaching strategy.

[0176] The conventional teaching equipment usually directly transmits unencrypted learning data through a general network, which has a risk of privacy leakage, and the teacher end can only view the original data, lacking visual analysis and interactive feedback functions. The embodiment guarantees data security through encryption and digital signature technology, improves the efficiency of teachers in combination with a structured display interface, and realizes low-delay feedback monitoring by using a polling mechanism, solving the problems of feedback lag and insufficient data security in the prior art. The embodiment realizes the secure transmission and efficient analysis of teaching data, enables teachers to quickly obtain visual analysis results of student status, and provides targeted guidance through various interactive methods. At the same time, the control device automatically captures and processes the teacher feedback information, ensuring the real-time and accuracy of teaching strategy adjustment, and effectively improving the response speed and decision quality of teaching intervention.

[0177] Optionally, with reference to Figure 9 , the present application further provides a control method of a teaching learning machine, based on the above-mentioned Figure 1 , the control method of the teaching learning machine further includes steps S700-S1100, wherein:

[0178] S700, if no teacher feedback information of the teacher terminal is monitored within a preset time length, a local emergency guidance mode is started;

[0179] S800, in the case of the local emergency guidance mode, historical learning behavior data of the student is acquired;

[0180] S900, a preset teaching problem solving scheme library is called through an interactive interface;

[0181] S1000, according to the historical learning behavior data, an adaptive auxiliary teaching scheme is matched in the teaching problem solving scheme library based on an optimal strategy;

[0182] S1100, the teaching guidance information is modulated based on the auxiliary teaching scheme, and the presentation mode or the difficulty level of the learning content is re-determined.

[0183] The local emergency guidance mode refers to a backup teaching strategy execution mechanism automatically triggered when no feedback from the teacher terminal is received within a preset time. The local emergency guidance mode can be realized by the linkage of a timer module and a feedback monitoring module, and the mode switching is triggered when the timer reaches the preset threshold and no valid feedback data packet is detected. The historical learning behavior data includes the student's past learning path record, knowledge point mastery trajectory, and interactive response characteristics, which can be extracted from the structured data stored in the learning state log through a database query interface. The teaching difficulty problem solution library stores the mapping relationship between typical learning obstacle scenarios and corresponding intervention measures, which can be constructed in the form of a knowledge graph. Each node is associated with problem type, solution strategy, and implementation parameters. The optimal strategy matching process uses a multi-factor weight evaluation algorithm, such as calculating the similarity between weak knowledge points in historical behavior data and strategy application conditions in the solution library to filter the highest matching solution.

[0184] When the teacher terminal fails to return feedback information in time due to network delay or operation delay, the control device automatically switches to the local emergency guidance mode. At this time, the student's learning record in the past three months is retrieved to analyze the error distribution law and interactive response characteristics on similar knowledge points. The pre-set auxiliary solutions in the teaching difficulty problem solution library include three types of visual explanation module, step-by-step guided exercise set, and knowledge point association graph. The control device preferentially matches the auxiliary solution containing dynamic demonstration and chart analysis according to the visual learning preference characteristics reflected in the student's historical data. The interactive interface immediately replaces the current text example with a three-dimensional model disassembly animation, and adjusts the exercise difficulty from the application level to the understanding level.

[0185] Compared with the prior art, the traditional teaching equipment can only pause the teaching process or repeat the original content when the teacher does not respond in time, resulting in the interruption of learning continuity. The embodiment establishes a local emergency decision mechanism, which can still select an adaptive strategy based on historical behavior characteristics in the case of teacher disconnection, maintaining the coherence of the teaching process. The prior art relies on a single linear feedback path, while the embodiment constructs a dual-channel teaching adjustment system to enable seamless connection when the main channel is blocked.

[0186] The embodiment effectively solves the problem of teaching strategy adjustment stagnation caused by teacher feedback delay, ensuring that students can still obtain content adaptation matching their learning characteristics without external intervention. The introduction of the local emergency mechanism avoids the passive waiting caused by the teacher's disconnection, which leads to distraction and knowledge gaps in traditional devices. The strategy matching driven by historical behavior data realizes the autonomous optimization of the teaching process. The pre-set structure of the solution library and the dynamic matching mechanism make the emergency teaching adjustment interpretable and targeted, avoiding the problem of repeated learning caused by blindly reducing the difficulty.

[0187] The application further provides a control device, which comprises a memory, a processor and a control program of a teaching learning machine stored in the memory and executable on the processor.

[0188] The memory is a non-volatile storage medium for storing the control program and learning state data, and can be implemented by a flash memory chip or a solid state disk, and is used for persistently storing program codes and collected student behavior data. The processor is an operation unit for executing program instructions to complete data analysis and decision, and can be implemented by a multi-core central processing chip, and is used for real-time processing of image information and calculation of learning confusion degree. The control program is a software program comprising multiple algorithm modules, and can be implemented by combining a machine learning model and a rule engine, and is used for dynamically adjusting teaching strategies and optimizing content presentation modes.

[0189] When the control program is running on the processor, the current learning progress and interactive feedback data of the student are acquired through an interactive interface, and adaptive learning content is pushed based on the data. Subsequently, a camera is started to collect facial images, expression states are determined by analyzing motion trajectories of key facial feature points, and attention degrees are calculated in combination with eye movement data. After the two parameters are weighted and fused, a learning confusion degree score is generated, based on which guide information is generated by calling a multi-level teaching strategy matrix, and content difficulty or presentation form is automatically adjusted. The updated learning data and guide strategies are encrypted and transmitted to a teacher terminal, and feedback information is continuously monitored to optimize subsequent decisions. If no timely teacher feedback is received, a local emergency mode is started to call historical data to match an auxiliary solution.

[0190] It is worth noting that, since the control device is based on the above-mentioned teaching learning machine control method, the embodiments of the control device include all the technical solutions of all the embodiments of the teaching learning machine control method, and achieve the same technical effects, which will not be repeated here.

[0191] The application further provides a teaching learning machine, which comprises the control device as described in the above embodiments.

[0192] Wherein, the teaching learning machine collects the facial image of the student in real time through the camera, and the feature point extraction and trajectory analysis are performed on the image by the processor running the control program to determine the expression state and attention level. The learning confusion degree is calculated by weighting and fusing the expression and attention data and combining the inherent difficulty coefficient of the learning content. According to the confusion degree level, the control program matches the corresponding teaching guidance scheme from the preset strategy matrix, such as converting the pure text into a mind map or adjusting the question difficulty. The updated learning state data is pushed to the teacher terminal after encryption, and the teacher feedback is obtained to optimize the strategy. If the teacher does not respond in time, the control device automatically calls the emergency teaching scheme matched with the historical behavior data to ensure the continuity of the learning process.

[0193] It is worth noting that, since the teaching learning machine of the present application is based on the above-mentioned control device, the embodiments of the teaching learning machine of the present application include all the technical solutions of all the embodiments of the above-mentioned control device, and the technical effects achieved are also exactly the same, which will not be repeated here.

[0194] It should be noted that in this paper, the term "including" "containing" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitation, the element defined by the sentence "including a…" does not exclude the presence of another identical element in the process, method, article or system including the element.

[0195] The above-mentioned embodiment serial numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0196] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server or network device, etc.) execute the method described in each embodiment of the present application.

[0197] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation made by using the content of the specification and drawings, or directly or indirectly applied to other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A control method of an educational learning machine including an interactive interface and a camera device, characterized by, The control method of the teaching learning machine comprises: obtaining current first learning state data of a student, pushing learning content corresponding to the first learning state data to the interactive interface, the first learning state data including learning progress and interactive feedback; controlling the camera to start, obtaining facial image information of the student, determining facial expression and attention degree corresponding to the facial image information; weighting and fusing the facial expression and the attention degree to determine the learning confusion degree of the student on the learning content; generating teaching guidance information corresponding to the learning confusion degree, and adjusting the presentation mode or difficulty level of the learning content according to the teaching guidance information; updating the first learning state data to second learning state data, pushing the second learning state data and the teaching guidance information to a teacher terminal, and continuously monitoring teacher feedback information of the teacher terminal; optimizing the teaching guidance information according to the teacher feedback information, and re-determining the presentation mode or difficulty level of the learning content; the weighting and fusing of the facial expression and the attention degree to determine the learning confusion degree of the student on the learning content comprises: assigning a first weight factor to the facial expression and a second weight factor to the attention degree, the sum of the first weight factor and the second weight factor being one; mapping the facial expression into an expression quantization value in a pre-set first numerical interval, and mapping the attention degree into an attention quantization value in a pre-set second numerical interval; multiplying the first weight factor and the expression quantization value to obtain a first weighted value, and multiplying the second weight factor and the attention quantization value to obtain a second weighted value; adding the first weighted value and the second weighted value to generate a first fusion score value; obtaining an inherent difficulty coefficient of the current learning content; determining the inherent difficulty coefficient as a gain coefficient, and gain processing the first fusion score value to generate a target fusion score value; comparing the target fusion score value with a plurality of difficulty threshold intervals one by one to determine the learning confusion degree.

2. The control method of an educational learning machine according to claim 1, characterized by, the control of the camera to start, the obtaining of the facial image information of the student, and the determination of the facial expression and the attention degree corresponding to the facial image information comprise: obtaining current ambient light parameters and orientation information of the student relative to the camera; adjusting the shooting parameters of the camera based on the ambient light parameters and the orientation information of the student relative to the camera; controlling the camera to obtain a plurality of facial image information of the student according to the shooting parameters; extracting key facial feature points of each facial image information one by one in the order of shooting time; sorting a plurality of key facial feature points to determine the motion trajectory and distribution state of the key facial feature points, and determining the facial expression corresponding to the motion trajectory and distribution state of the key facial feature points; extracting the position change trajectory of the eyeball pupil and the eyelid opening degree in a plurality of the facial image information, and determining the visual focus degree and the blink frequency; The attention degree is determined by weighted fusion of the line-of-sight focus degree and the blink frequency, and the attention degree includes full concentration, attention concentration, and attention dispersion.

3. The control method of an educational learning machine according to claim 2, wherein The motion trajectory and distribution state of the key facial feature points are determined by sorting the key facial feature points, and a facial expression corresponding to the motion trajectory and distribution state of the key facial feature points is determined. The key facial feature points are subjected to coordinate normalization processing. The coordinate change amount of the same key facial feature point in a continuous time sequence is calculated to generate a motion vector of each key facial feature point. The motion vectors of the key facial feature points are summarized to determine the comprehensive motion trend of all the key facial feature points and generate the distribution state. A motion trajectory mode representing the overall facial muscle motion is generated according to the motion vector and the distribution state. The motion trajectory mode is matched with a preset typical expression template library, and the typical expression template library stores standard motion trajectory modes corresponding to four basic expressions, i.e., happy, calm, confused, and bored. The similarity between the motion trajectory and each standard motion trajectory mode in the typical expression template library is calculated. The expression corresponding to the standard motion trajectory mode with the highest similarity is selected as the finally determined facial expression. The line-of-sight focus degree and the blink frequency are determined by extracting the position change trajectory of the eyeball pupil and the eyelid opening degree in the facial image information. The eyeball region is located in each frame of facial image information, and the center coordinates of the pupil are identified. The movement path of the pupil center coordinates in continuous multiple frames of facial image information is tracked to form a position change trajectory. The total length of the position change trajectory in a unit time is calculated, and if the length is lower than a first threshold, it is determined that the line-of-sight focus degree is high, and if the length is higher than a second threshold, it is determined that the line-of-sight focus degree is low. The vertical distance between the upper eyelid and the lower eyelid in each frame of image is extracted. If it is monitored that the eyelid opening degree changes from higher than a threshold to lower than a threshold and then recovers in continuous two frames of image, it is counted as one blink event. The total number of blink events occurring in a unit time window is counted to determine the blink frequency.

4. The control method of an educational learning machine according to claim 1, wherein The teaching guidance information includes a strategy identifier. The teaching guidance information corresponding to the learning confusion degree is generated, and the presentation mode or the difficulty level of the learning content is adjusted according to the teaching guidance information, including: A multi-level teaching strategy matrix is preset, and the multi-level teaching strategy matrix takes the learning confusion degree level as the row and the knowledge point type as the column. One or more candidate teaching guidance strategies are located in the multi-level teaching strategy matrix according to the learning confusion degree level and the knowledge point type to which the learning content belongs. If the strategy identifier indicates adjustment of the presentation mode, the first execution parameter of the teaching guidance information is analyzed, and the current pure text discussion is converted into a structured mind map, or a segment of demonstration animation is inserted, or a key formula is highlighted on the interactive interface. If the policy identifier indicates adjusting the difficulty level, according to a target difficulty value in the first execution parameter, an alternative question set meeting the target difficulty value is screened out from a preset question bank, and the alternative question set is pushed to the student through an interactive interface.

5. The control method of an educational learning machine according to claim 4, wherein The one or more candidate teaching guidance strategies are located in the multi-level teaching strategy matrix according to the learning confusion level and the knowledge point type to which the content being learned belongs, and the locating includes: quantifying the learning confusion level into an integer value as a row index of the matrix; encoding the knowledge point type to map a category identifier as a column index of the matrix; locating an initial strategy unit in the multi-level teaching strategy matrix according to the row index and the column index, the initial strategy unit including a basic strategy set; calling the historical strategy application effect record of the student to calculate the historical average efficiency of each strategy in the basic strategy set; adding a time decay coefficient to the historical average efficiency of each strategy to determine a current timeliness weight value of each strategy; sorting all strategies in the basic strategy set in descending order according to the current timeliness weight value to select the top several strategies as the final candidate teaching guidance strategies.

6. The control method of an educational learning machine according to claim 1, wherein The first learning state data is updated to second learning state data, the second learning state data and the teaching guidance information are pushed to a teacher terminal, and teacher feedback information of the teacher terminal is continuously monitored, and the updating includes: encrypting and packaging the first learning state data, the learning confusion level, the teaching guidance information and the adjusted learning content segment into a second learning state data package after summarizing them; linking the second learning state data package to a data package content containing a timestamp and a digital signature, and pushing it to the corresponding teacher terminal through a secure communication link; controlling the teacher terminal side to analyze and visually display the data package content, generate and provide a teacher feedback interface containing preset options and a custom input box; starting one or more background monitoring services to continuously monitor whether there is a feedback data package returned from the network port of the teacher terminal in a polling manner; decrypting and analyzing the received feedback data package to extract the preset feedback instructions selected by the teacher, the text annotation information input by the teacher or the supplementary teaching resource file uploaded by the teacher, and generating the teacher feedback information.

7. The control method of an educational learning machine according to claim 1, wherein The control method of the teaching learning machine further includes: if no teacher feedback information of the teacher terminal is monitored within a preset time length, a local emergency guidance mode is started; in the case of the local emergency guidance mode, the historical learning behavior data of the student is obtained; a preset teaching problem solving scheme library is called through an interactive interface; an adaptive auxiliary teaching scheme is matched in the teaching problem solving scheme library based on the optimal strategy according to the historical learning behavior data; the teaching guidance information is modulated based on the auxiliary teaching scheme, and the presentation mode or the difficulty level of the learning content is re-determined.

8. A control device for an educational learning machine, characterized by comprising: The control device comprises a memory, a processor, and a control program of the teaching learning machine stored on the memory and executable on the processor, and the control program of the teaching learning machine is configured to implement the control method of the teaching learning machine according to any one of claims 1 to 7.

9. A teaching-learning machine characterized by comprising: The control device according to claim 8 is included.

Images