An AI-based teaching interaction system

Abstract

The present invention relates to the technical field of teaching interaction, and discloses an artificial intelligence-based teaching interaction system, including: an image acquisition module, which is used to obtain classroom image data during class in real time and preprocess it; a concentration calculation module, which is used to analyze the concentration of each student during n periods of class according to the preprocessed classroom image data; a concentration correction module, which is used to correct the output of the concentration calculation module according to the output of the concentration calculation module. The present invention obtains the image data of each student in the classroom at each time period in real time, then calculates the concentration of each student at each time period, and corrects the concentration of other students according to the influence of the student with the lowest concentration in the classroom on other students. Compared with general concentration calculation methods, the calculation of the concentration of students is more accurate, thereby improving the accuracy of the heat map generated later and facilitating the teacher to adjust their teaching style and rhythm.

Description

Technical Field

[0001] The present invention relates to the technical field of teaching interaction, and more specifically, it relates to a teaching interaction system based on artificial intelligence. Background Art

[0002] In the traditional education system, classroom teaching usually follows a relatively fixed pattern, that is, teachers explain according to the established teaching syllabus and plan, while students are expected to participate in the learning process with a consistent attention level. However, this pattern ignores the significant differences among students (different learning bases, personal interests, energy states, and classroom participation attitudes), and these factors will directly affect the concentration level of students in class. The level of concentration not only affects the absorption efficiency of knowledge by students, but also is an important indicator for evaluating the teaching interaction effect and adjusting teaching strategies.

[0003] A common method is to collect students' facial images and then analyze them to obtain the concentration of students. However, the above method does not consider that the actions and sounds made by students when their concentration drops will affect the concentration of surrounding students. Therefore, a teaching interaction system based on artificial intelligence is proposed. Summary of the Invention

[0004] The present invention provides a teaching interaction system based on artificial intelligence to solve the technical problems in the above background art.

[0005] The present invention provides a teaching interaction system based on artificial intelligence, including:

[0006] An image acquisition module, which is used to obtain classroom image data in real time during class and preprocess it;

[0007] A concentration calculation module, which is used to analyze the concentration of each student during n periods in class according to the preprocessed classroom image data;

[0008] A concentration correction module, which is used to correct the output of the concentration calculation module according to the output of the concentration calculation module;

[0009] A feedback module, which is used to feedback the concentration situation of each student during n periods to the teacher.

[0010] Furthermore, the image acquisition module includes a camera unit and an image preprocessing unit;

[0011] Among them, the camera unit is constructed based on a camera device to obtain classroom image data in real time; the image preprocessing unit is used to preprocess the classroom image data, and it includes: marking and extracting key frames.

[0012] Furthermore, the specific steps for the image preprocessing unit to preprocess the classroom image data are as follows:

[0013] Step S501: Uniformly extract m key frames from the classroom image data, where m is a custom parameter;

[0014] Step S502: Use the YOLO algorithm to mark each student in each key frame to generate action image data, and the size of each action image data is the same;

[0015] Step S503: If the resolution of the action image data in a key frame is lower than the preset resolution threshold, then select the action image data at the same position in the adjacent frame of the key frame in the classroom image data for replacement, and the preset resolution threshold is a custom parameter.

[0016] Furthermore, the concentration calculation module includes a first hidden layer, a second hidden layer, and a classifier;

[0017] The first hidden layer includes m hidden units;

[0018] The i-th action image data of one student is input into the i-th hidden unit, and the first updated feature is output;

[0019] All the first updated features output by the first hidden layer are input into the second hidden layer, and the second updated feature is output;

[0020] The second updated feature is input into the classifier, and the concentration of the student in n time periods during class is output;

[0021] 1 ≤ i ≤ m, and n is a custom parameter.

[0022] Furthermore, the m hidden units of the first hidden layer of the concentration calculation module are all constructed based on a convolutional neural network, and the second hidden layer is constructed based on a long short-term memory network.

[0023] Furthermore, the training samples of the concentration calculation module are specifically as follows:

[0024] Select L students with similar grades to take a test course at the same time, and obtain the classroom image data of the L students during the test course. Then preprocess it to generate L test action image data sets. Each test action image data set contains m key frames, and each test action image data set is the sample data of a training sample;

[0025] The test course is divided into n unrelated contents, which are played in sequence, and the n test courses and n time periods are set in one-to-one correspondence;

[0026] After the test course, questions are set according to the n contents of the test course, and the score of each student for each content is the sample label of a training sample;

[0027] L is a custom parameter.

[0028] Furthermore, the calculation method for the focus correction module to correct the output of the focus calculation module is as follows:

[0029] ;

[0030] In the formula, represents the focus of a certain student at a certain time period output by the focus calculation module; represents the corrected focus of this student at this time period; IF represents the focus influence coefficient of the student with the lowest focus in the classroom; d represents the distance between this student and the student with the lowest focus in the classroom; LPDF represents the difference between the attention of the student with the lowest focus in the classroom and his own focus in the previous key frame.

[0031] Furthermore, the calculation formula of IF is as follows:

[0032] ;

[0033] In the formula, BaseInfluence represents the average influence constant of the student with the lowest focus in the classroom; HistoricalBehavior represents the historical behavior factor of the student with the lowest focus in the classroom; and represent the first and second weight parameters; BaseInfluence, HistoricalBehavior, and are all custom parameters.

[0034] Furthermore, the feedback module generates a heat map based on the focus of each student in each time period and transmits it to the teacher's client in real time.

[0035] Furthermore, the calculation formula of the second hidden layer of the focus calculation module includes:

[0036] ;

[0037] ;

[0038] ;

[0039] ;

[0040] ;

[0041] ;

[0042] In the formula, 、 , , , and respectively represent the input gate, forget gate, output gate, candidate memory state, memory state, and hidden state of the second hidden layer of the concentration calculation module; among them, , and respectively represent the first weight parameter, second weight parameter, and bias parameter corresponding to the input gate of the second hidden layer of the concentration calculation module; , and respectively represent the first weight parameter, second weight parameter, and bias parameter corresponding to the forget gate of the second hidden layer of the concentration calculation module; , and respectively represent the first weight parameter, second weight parameter, and bias parameter corresponding to the output gate of the second hidden layer of the concentration calculation module; , and respectively represent the first weight parameter, second weight parameter, and bias parameter corresponding to the candidate memory state of the second hidden layer of the concentration calculation module; represents the sigmoid function; tanh represents the hyperbolic tangent function; represents the dot product operation; represents the input of the second hidden layer of the concentration calculation module; represents the previous hidden state.

[0043] The beneficial effects of the present invention are as follows: real-time acquisition of image data of each student in the classroom at each time period, then calculation of the concentration of each student at each time period, and correction of the concentration of other students according to the influence of the student with the lowest concentration in the classroom on other students. Compared with general concentration calculation methods, the calculation of students' concentration is more accurate, thereby improving the accuracy of the heat map generated later and facilitating the teacher to adjust their teaching style and rhythm. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is the overall flow block diagram of the present invention;

[0045] Figure 2 is the flow block diagram of the image preprocessing unit of the present invention for preprocessing classroom image data;

[0046] Figure 3 is one of the schematic diagrams of the heat map of students' concentration of the present invention;

[0047] Figure 4This is the second schematic diagram of the heat map of student concentration of the present invention.

[0048] In the figure: 10, image acquisition module; 20, concentration calculation module; 30, concentration correction module; 40, feedback module;

[0049] 101. Camera unit; 102. Image preprocessing unit;

[0050] 1. The first area; 2. The second area; 3. The third area; 4. The fourth area. DETAILED DESCRIPTION

[0051] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0052] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in one or more embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0053] The purpose of the embodiment of the present invention is that when the teacher is teaching, since each student has different knowledge mastery foundation, mental state and learning attitude in class, different students will have different concentrations in class, and the concentration of students in class will directly affect their learning effect. The teacher cannot directly grasp the concentration of all students in class, which makes it impossible for the teacher to adjust his teaching style and rhythm in time according to the concentration of students, thereby reducing the learning efficiency of the entire class. Therefore, an artificial intelligence-based teaching interactive system is proposed, such as Figure 1-Figure 2 As shown, it includes: an image acquisition module 10, a concentration calculation module 20, a concentration correction module 30 and an automatic evaluation module 40.

[0054] First, it is necessary to obtain image data of students in class to analyze their concentration. Therefore, the image acquisition module 10 is used to obtain classroom image data in real time and pre-process it. The classroom image data includes images of all students in the classroom.

[0055] The image acquisition module 10 includes a camera unit 101 and an image preprocessing unit 102; preferably, the camera unit 101 is arranged at the top of the podium, so that image information of all students can be captured;

[0056] Among them, the camera unit 101 is constructed based on the camera equipment and obtains classroom image data in real time; the image preprocessing unit 102 is used to preprocess the classroom image data, which includes: marking and extracting key frames.

[0057] The specific steps of the image preprocessing unit 102 preprocessing the classroom image data are as follows:

[0058] Step S501, uniformly extract m key frames from the classroom image data, where m is a custom parameter;

[0059] Step S502, using the yolo algorithm to mark each student in each key frame to generate action image data, each action image data has the same size, the yolo algorithm belongs to the prior art and will not be described in detail here;

[0060] Step S503: if the resolution of action image data in a key frame is lower than a preset resolution threshold, the action image data at the same position in the adjacent frame of the key frame in the classroom image data is selected for replacement, and the preset resolution threshold is a custom parameter.

[0061] After preprocessing the classroom image data during class, the concentration of students can be analyzed. However, the concentration of students in a class will not remain unchanged, so it is necessary to analyze the concentration of each time period. Therefore, the concentration calculation module 20 is used to analyze the concentration of each student in n time periods during class according to the preprocessed classroom image data;

[0062] The concentration calculation module 20 includes a first hidden layer, a second hidden layer and a classifier;

[0063] The first hidden layer includes m hidden units;

[0064] The i-th action image data of one of the students is input into the i-th hidden unit, and a first updated feature is output, which includes the head orientation information of the student and whether the student is in a blinking state;

[0065] All first updated features output by the first hidden layer are input into the second hidden layer, and the second updated features are output;

[0066] Second, update the feature input classifier and output the student's concentration in n time periods during class. Divide the class time (usually 45 minutes) into several segments, preferably set to 9, and analyze the concentration of each time period to better understand the changes in the student's concentration in each time period.

[0067] 1≤i≤m, n is a custom parameter.

[0068] In addition, the m hidden units of the first hidden layer of the concentration calculation module 20 are all constructed based on a convolutional neural network, and the second hidden layer is constructed based on a long short-term memory network.

[0069] The calculation formula of the second hidden layer of the concentration calculation module 20 includes:

[0070] ;

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] ;

[0076] In the formula, , , , , and represent the input gate, forget gate, output gate, candidate memory state, memory state and hidden state of the second hidden layer of the concentration calculation module 20 respectively; wherein, , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to an input gate of a second hidden layer of the concentration calculation module 20; , and Respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to a forget gate of a second hidden layer of the concentration calculation module 20; , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to an output gate of a second hidden layer of the concentration calculation module 20; , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to the candidate memory state of the second hidden layer of the concentration calculation module 20; represents the sigmoid function; tanh represents the hyperbolic tangent function; Represents the dot product operation; represents the input of the second hidden layer of the concentration calculation module 20; Represents the previous hidden state.

[0077] Before the concentration calculation module 20 is put into use, it needs to be trained to ensure that its accuracy meets the application requirements. Therefore, the training samples of the concentration calculation module 20 are as follows:

[0078] Select L students with similar grades to take a test course at the same time, and obtain classroom image data of L students when taking the test course, and then preprocess them to generate L test action image datasets, each of which contains m key frames, and each test action image dataset is the sample data of a training sample;

[0079] The test course is divided into n unrelated contents, which are played in sequence, and n test courses and n time periods are set in one-to-one correspondence;

[0080] After the test course, test questions are given according to the n contents of the test course. The score of each student for each content is the sample label of a training sample.

[0081] L is a custom parameter.

[0082] The above content provides a method for analyzing the concentration of students in each period of time through the head orientation information and blinking status information of students. However, it does not take into account the situation that when a student's concentration decreases, talking or violent movements will affect the concentration of surrounding students. At this time, even if the student's head is still facing the teacher or the book, his concentration will inevitably decrease. Therefore, the concentration correction module 30 corrects the concentration of each student in n periods of time during class output by the concentration calculation module 20 according to the influence of the concentration between students;

[0083] The calculation method of the concentration correction module 30 correcting the output of the concentration calculation module 20 is specifically as follows:

[0084] ;

[0085] In the formula, Indicates the concentration of a student in a certain period of time output by the concentration calculation module 20; represents the corrected concentration of the student in this period; IF represents the concentration influence coefficient of the student with the lowest concentration in the classroom; d represents the distance between the student and the student with the lowest concentration in the classroom; LPDF represents the difference between the concentration of the student with the lowest concentration in the classroom and the student's own concentration in the previous key frame;

[0086] The calculation formula of IF is as follows:

[0087] ;

[0088] In the formula, BaseInfluence represents the average influence constant of the least focused students in the classroom; HistoricalBehavior represents the historical behavior factor of the least focused students in the classroom; and Represents the first and second weight parameters; BaseInfluence, HistoricalBehavior, and These are all custom parameters, and the concentration of each student is corrected through the above method.

[0089] After the concentration of each student in each time period is corrected, the concentration of each time period needs to be fed back to the teacher in real time to ensure that the teacher can obtain the concentration of each student in each time period in the class, so that the teacher can adjust his or her teaching style and rhythm. Therefore, the feedback module 40 is used to feed back the concentration of each student in n time periods to the teacher;

[0090] The feedback module 40 generates a heat map according to the concentration of each student in each time period, and transmits it to the teacher's client in real time, dividing the students' concentration from low to high into four areas, namely the first area 1, the second area 2, the third area 3 and the fourth area 4. Figure 3-Figure 4 shown.

[0091] The above describes an embodiment of the present embodiment, but the present embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present embodiment, ordinary technicians in this field can also make many forms, all of which are protected by the present embodiment.

Claims

1. An interactive teaching system based on artificial intelligence, characterized in that: include: An image acquisition module (10) is used to acquire classroom image data during class in real time and pre-process the data; A concentration calculation module (20), which is used to analyze the concentration of each student in n time periods during class according to the pre-processed classroom image data; A concentration correction module (30), which is used to correct the concentration of each student in n time periods during class output by the concentration calculation module (20) according to the influence of the concentration between students; The calculation method of the concentration correction module (30) correcting the output of the concentration calculation module (20) is specifically as follows: ； In the formula, represents the concentration of a student in a certain period of time output by the concentration calculation module (20); represents the corrected concentration of the student in this period; IF represents the concentration influence coefficient of the least focused student in the classroom; d represents the distance between the student and the least focused student in the classroom; LPDF represents the difference between the concentration of the least focused student in the classroom and the student's own concentration in the previous key frame; The calculation formula of IF is as follows: ； In the formula, BaseInfluence represents the average influence constant of the least focused students in the classroom; HistoricalBehavior represents the historical behavior factor of the least focused students in the classroom; and Represents the first and second weight parameters; BaseInfluence, HistoricalBehavior, and All are custom parameters; The feedback module (40) is used to provide the teacher with feedback on the concentration of each student in n time periods.

2. The teaching interactive system based on artificial intelligence according to claim 1, characterized in that: The image acquisition module (10) comprises a camera unit (101) and an image preprocessing unit (102); The camera unit (101) is constructed based on a camera device and acquires classroom image data in real time; the image preprocessing unit (102) is used to preprocess the classroom image data, including marking and extracting key frames.

3. The teaching interactive system based on artificial intelligence according to claim 2, characterized in that: The specific steps of the image preprocessing unit (102) preprocessing the classroom image data are as follows: Step S501, uniformly extract m key frames from the classroom image data, where m is a custom parameter; Step S502, using the YOLO algorithm to mark each student in each key frame to generate action image data, each action image data has the same size; Step S503: if the resolution of action image data in a key frame is lower than a preset resolution threshold, the action image data at the same position in the adjacent frame of the key frame in the classroom image data is selected for replacement, and the preset resolution threshold is a custom parameter.

4. The teaching interactive system based on artificial intelligence according to claim 1, characterized in that: The concentration calculation module (20) includes a first hidden layer, a second hidden layer and a classifier; The first hidden layer includes m hidden units; The i-th action image data of one of the students is input into the i-th hidden unit, and the first updated feature is output; All first updated features output by the first hidden layer are input into the second hidden layer, and the second updated features are output; Second, update the feature input classifier and output the student's concentration in n time periods during class; 1≤i≤m, n is a custom parameter.

5. The teaching interactive system based on artificial intelligence according to claim 4 is characterized in that: The m hidden units of the first hidden layer of the concentration calculation module (20) are all constructed based on a convolutional neural network, and the second hidden layer is constructed based on a long short-term memory network.

6. The teaching interactive system based on artificial intelligence according to claim 4, characterized in that: The training samples of the concentration calculation module (20) are as follows: Select L students with similar grades to take a test course at the same time, and obtain classroom image data of L students when taking the test course, and then preprocess them to generate L test action image datasets, each of which contains m key frames, and each test action image dataset is the sample data of a training sample; The test course is divided into n unrelated contents, which are played in sequence, and n test courses and n time periods are set in one-to-one correspondence; After the test course, test questions are given according to the n contents of the test course. The score of each student for each content is the sample label of a training sample. L is a custom parameter.

7. The teaching interactive system based on artificial intelligence according to claim 1, characterized in that: The feedback module (40) generates a heat map according to the concentration of each student in each time period, and transmits it to the teacher's client in real time.

8. The teaching interactive system based on artificial intelligence according to claim 5, characterized in that: The calculation formula of the second hidden layer of the concentration calculation module (20) includes: ； ； ； ； ； ； In the formula, , , , , and They respectively represent the input gate, forget gate, output gate, candidate memory state, memory state and hidden state of the second hidden layer of the concentration calculation module (20); wherein, , and Respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to an input gate of a second hidden layer of a concentration calculation module (20); , and Respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to a forget gate of a second hidden layer of a concentration calculation module (20); , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to an output gate of a second hidden layer of a concentration calculation module (20); , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to the candidate memory state of the second hidden layer of the concentration calculation module (20); represents the sigmoid function; tanh represents the hyperbolic tangent function; Represents the dot product operation; represents the input of the second hidden layer of the concentration calculation module (20); Represents the previous hidden state.

Images