Teaching interaction system based on artificial intelligence

Abstract

The invention relates to the technical field of teaching interaction, and discloses a teaching interaction system based on artificial intelligence, and the system comprises an image collection module which is used for obtaining classroom image data in a class in real time, and carrying out the preprocessing of the classroom image data; the concentration degree calculation module is used for analyzing the concentration degree of each student in n time periods in class according to the preprocessed classroom image data; and the concentration correction module is used for correcting the output of the concentration calculation module according to the output of the concentration calculation module. According to the method, the image data of each student in the classroom in each time period is acquired in real time, then the concentration degree of each student in each time period is calculated, and the concentration degrees of other students are corrected according to the influence of the student with the lowest concentration degree in the classroom on other students, so that compared with a general concentration degree calculation method, the method has the advantage that the calculation efficiency is improved. And the concentration degree of the students is calculated more accurately, so that the accuracy of the popularity map generated in the later period is improved, and a teacher can conveniently adjust the teaching style and rhythm of himself / herself.

Description

Technical Field

[0001] The present invention relates to the field of teaching interaction technology, and more specifically, to a teaching interaction system based on artificial intelligence. Background Art

[0002] In the traditional education system, classroom teaching usually follows a relatively fixed pattern, that is, teachers explain according to the established syllabus and plan, and students are expected to participate in the learning process with a consistent level of attention. However, this model ignores the significant differences among students (different learning foundations, personal interests, energy states and classroom participation attitudes). These factors will directly affect students' concentration in class. The level of concentration is not only related to students' efficiency in absorbing knowledge, but also an important indicator for evaluating the effectiveness of teaching interaction and adjusting teaching strategies.

[0003] A common method is to collect students' facial images and then analyze them to obtain their concentration. However, the above method does not take into account that the actions and sounds made by students when their concentration decreases will affect the concentration of surrounding students. Therefore, an artificial intelligence-based teaching interaction system is proposed. Summary of the invention

[0004] The present invention provides an artificial intelligence-based teaching interactive system to solve the technical problems in the above-mentioned background technology.

[0005] The present invention provides a teaching interactive system based on artificial intelligence, comprising: An image acquisition module is used to obtain classroom image data in real time and pre-process it; A concentration calculation module is used to analyze the concentration of each student in n time periods during class based on the preprocessed classroom image data; A concentration correction module, which is used to correct the output of the concentration calculation module according to the output of the concentration calculation module; The feedback module is used to provide the teacher with feedback on the concentration of each student in n time periods.

[0006] Furthermore, the image acquisition module includes a camera unit and an image preprocessing unit; Among them, the camera unit is constructed based on the camera equipment to obtain classroom image data in real time; the image preprocessing unit is used to preprocess the classroom image data, which includes: marking and extracting key frames.

[0007] Furthermore, the specific steps of the image preprocessing unit preprocessing the classroom image data are as follows: Step S501, uniformly extract m key frames from the classroom image data, where m is a custom parameter; Step S502, using the YOLO algorithm to mark each student in each key frame to generate action image data, each action image data has the same size; Step S503: if the resolution of action image data in a key frame is lower than a preset resolution threshold, the action image data at the same position in the adjacent frame of the key frame in the classroom image data is selected for replacement, and the preset resolution threshold is a custom parameter.

[0008] Further, the concentration calculation module includes a first hidden layer, a second hidden layer and a classifier; The first hidden layer includes m hidden units; The i-th action image data of one of the students is input into the i-th hidden unit, and the first updated feature is output; All first updated features output by the first hidden layer are input into the second hidden layer, and the second updated features are output; Second, update the feature input classifier and output the student's concentration in n time periods during class; 1≤i≤m, n is a custom parameter.

[0009] Furthermore, the m hidden units of the first hidden layer of the concentration calculation module are all constructed based on a convolutional neural network, and the second hidden layer is constructed based on a long short-term memory network.

[0010] Furthermore, the training samples of the concentration calculation module are as follows: Select L students with similar grades to take a test course at the same time, and obtain classroom image data of L students when taking the test course, and then preprocess them to generate L test action image datasets, each of which contains m key frames, and each test action image dataset is the sample data of a training sample; The test course is divided into n unrelated contents, which are played in sequence, and n test courses and n time periods are set in one-to-one correspondence; After the test course, test questions are given according to the n contents of the test course. The score of each student for each content is the sample label of a training sample. L is a custom parameter.

[0011] Furthermore, the calculation method of the concentration correction module to correct the output of the concentration calculation module is specifically as follows: ; In the formula, It indicates the concentration of a student in a certain period of time output by the concentration calculation module; It represents the corrected concentration of the student in this period; IF represents the concentration influence coefficient of the student with the lowest concentration in the classroom; d represents the distance between the student and the student with the lowest concentration in the classroom; LPDF represents the difference between the concentration of the student with the lowest concentration in the classroom and the student's own concentration in the previous key frame.

[0012] Furthermore, the calculation formula of IF is as follows: ; In the formula, BaseInfluence represents the average influence constant of the least focused students in the classroom; HistoricalBehavior represents the historical behavior factor of the least focused students in the classroom; and Represents the first and second weight parameters; BaseInfluence, HistoricalBehavior, and These are custom parameters.

[0013] Furthermore, the feedback module generates a heat map according to the concentration of each student in each period and transmits it to the teacher's client in real time.

[0014] Furthermore, the calculation formula of the second hidden layer of the concentration calculation module includes: ; ; ; ; ; ; In the formula, , , , , and They represent the input gate, forget gate, output gate, candidate memory state, memory state and hidden state of the second hidden layer of the concentration calculation module respectively; , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to an input gate of a second hidden layer of a concentration calculation module; , and Respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to a forget gate of a second hidden layer of a concentration calculation module; , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to an output gate of a second hidden layer of a concentration calculation module; , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to the candidate memory state of the second hidden layer of the concentration calculation module; represents the sigmoid function; tanh represents the hyperbolic tangent function; Represents the dot product operation; Represents the input of the second hidden layer of the concentration calculation module; Represents the previous hidden state.

[0015] The beneficial effects of the present invention are: real-time acquisition of image data of each student in the classroom in each time period, then calculation of the concentration of each student in each time period, and correction of the concentration of other students according to the influence of the least focused student in the classroom on other students. Compared with the general concentration calculation method, the calculation of student concentration is more accurate, thereby improving the accuracy of the heat map generated later, making it easier for teachers to adjust their teaching style and rhythm. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is the overall flow chart of the present invention; Figure 2 It is a flowchart of the image preprocessing unit of the present invention preprocessing classroom image data; Figure 3 It is one of the heat map schematic diagrams of the student concentration of the present invention; Figure 4 This is the second schematic diagram of the heat map of student concentration of the present invention.

[0017] In the figure: 10, image acquisition module; 20, concentration calculation module; 30, concentration correction module; 40, feedback module; 101. Camera unit; 102. Image preprocessing unit; 1. The first area; 2. The second area; 3. The third area; 4. The fourth area. DETAILED DESCRIPTION

[0018] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0019] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in one or more embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0020] The purpose of the embodiment of the present invention is that when the teacher is teaching, since each student has different knowledge mastery foundation, mental state and learning attitude in class, different students will have different concentrations in class, and the concentration of students in class will directly affect their learning effect. The teacher cannot directly grasp the concentration of all students in class, which makes it impossible for the teacher to adjust his teaching style and rhythm in time according to the concentration of students, thereby reducing the learning efficiency of the entire class. Therefore, an artificial intelligence-based teaching interactive system is proposed, such as Figure 1-Figure 2 As shown, it includes: an image acquisition module 10, a concentration calculation module 20, a concentration correction module 30 and an automatic evaluation module 40.

[0021] First, it is necessary to obtain image data of students in class to analyze their concentration. Therefore, the image acquisition module 10 is used to obtain classroom image data in real time and pre-process it. The classroom image data includes images of all students in the classroom. The image acquisition module 10 includes a camera unit 101 and an image preprocessing unit 102; preferably, the camera unit 101 is arranged at the top of the podium, so that image information of all students can be captured; Among them, the camera unit 101 is constructed based on the camera equipment and obtains classroom image data in real time; the image preprocessing unit 102 is used to preprocess the classroom image data, which includes: marking and extracting key frames.

[0022] The specific steps of the image preprocessing unit 102 preprocessing the classroom image data are as follows: Step S501, uniformly extract m key frames from the classroom image data, where m is a custom parameter; Step S502, using the yolo algorithm to mark each student in each key frame to generate action image data, each action image data has the same size, the yolo algorithm belongs to the prior art and will not be described in detail here; Step S503: if the resolution of action image data in a key frame is lower than a preset resolution threshold, the action image data at the same position in the adjacent frame of the key frame in the classroom image data is selected for replacement, and the preset resolution threshold is a custom parameter.

[0023] After preprocessing the classroom image data during class, the concentration of students can be analyzed. However, the concentration of students in a class will not remain unchanged, so it is necessary to analyze the concentration of each time period. Therefore, the concentration calculation module 20 is used to analyze the concentration of each student in n time periods during class according to the preprocessed classroom image data; The concentration calculation module 20 includes a first hidden layer, a second hidden layer and a classifier; The first hidden layer includes m hidden units; The i-th action image data of one of the students is input into the i-th hidden unit, and the first updated feature is output. The first updated feature includes the head orientation information of the student and whether the student is in a blinking state; All first updated features output by the first hidden layer are input into the second hidden layer, and the second updated features are output; Second, update the feature input classifier and output the student's concentration in n time periods during class. Divide the class time (usually 45 minutes) into several segments, preferably set to 9, and analyze the concentration of each time period to better understand the changes in the student's concentration in each time period. 1≤i≤m, n is a custom parameter.

[0024] In addition, the m hidden units of the first hidden layer of the concentration calculation module 20 are all constructed based on a convolutional neural network, and the second hidden layer is constructed based on a long short-term memory network.

[0025] The calculation formula of the second hidden layer of the concentration calculation module 20 includes: ; ; ; ; ; ; In the formula, , , , , and represent the input gate, forget gate, output gate, candidate memory state, memory state and hidden state of the second hidden layer of the concentration calculation module 20 respectively; wherein, , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to an input gate of a second hidden layer of the concentration calculation module 20; , and Respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to a forget gate of a second hidden layer of the concentration calculation module 20; , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to an output gate of a second hidden layer of the concentration calculation module 20; , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to the candidate memory state of the second hidden layer of the concentration calculation module 20; represents the sigmoid function; tanh represents the hyperbolic tangent function; Represents the dot product operation; represents the input of the second hidden layer of the concentration calculation module 20; Represents the previous hidden state.

[0026] Before the concentration calculation module 20 is put into use, it needs to be trained to ensure that its accuracy meets the application requirements. Therefore, the training samples of the concentration calculation module 20 are as follows: Select L students with similar grades to take a test course at the same time, and obtain classroom image data of L students when taking the test course, and then preprocess them to generate L test action image datasets, each of which contains m key frames, and each test action image dataset is the sample data of a training sample; The test course is divided into n unrelated contents, which are played in sequence, and n test courses and n time periods are set in one-to-one correspondence; After the test course, test questions are given according to the n contents of the test course. The score of each student for each content is the sample label of a training sample. L is a custom parameter.

[0027] The above content provides a method for analyzing the concentration of students in each period of time through the head orientation information and blinking status information of students. However, it does not take into account the situation that when a student's concentration decreases, talking or violent movements will affect the concentration of surrounding students. At this time, even if the student's head is still facing the teacher or the book, his concentration will inevitably decrease. Therefore, the concentration correction module 30 corrects the concentration of each student in n periods of time during class output by the concentration calculation module 20 according to the influence of the concentration between students; The calculation method of the concentration correction module 30 correcting the output of the concentration calculation module 20 is specifically as follows: ; In the formula, Indicates the concentration of a student in a certain period of time output by the concentration calculation module 20; represents the corrected concentration of the student in this period; IF represents the concentration influence coefficient of the student with the lowest concentration in the classroom; d represents the distance between the student and the student with the lowest concentration in the classroom; LPDF represents the difference between the concentration of the student with the lowest concentration in the classroom and the student's own concentration in the previous key frame; The calculation formula of IF is as follows: ; In the formula, BaseInfluence represents the average influence constant of the least focused students in the classroom; HistoricalBehavior represents the historical behavior factor of the least focused students in the classroom; and Represents the first and second weight parameters; BaseInfluence, HistoricalBehavior, and These are all custom parameters, and the concentration of each student is corrected through the above method.

[0028] After the concentration of each student in each time period is corrected, the concentration of each time period needs to be fed back to the teacher in real time to ensure that the teacher can obtain the concentration of each student in each time period in the class, so that the teacher can adjust his or her teaching style and rhythm. Therefore, the feedback module 40 is used to feed back the concentration of each student in n time periods to the teacher; The feedback module 40 generates a heat map according to the concentration of each student in each time period, and transmits it to the teacher's client in real time, dividing the students' concentration from low to high into four areas, namely the first area 1, the second area 2, the third area 3 and the fourth area 4. Figure 3-Figure 4 shown.

[0029] The above describes an embodiment of the present embodiment, but the present embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present embodiment, ordinary technicians in this field can also make many forms, all of which are within the protection of the present embodiment.

Claims

1. An interactive teaching system based on artificial intelligence, characterized in that: include: An image acquisition module (10) is used to acquire classroom image data during class in real time and pre-process the data; A concentration calculation module (20), which is used to analyze the concentration of each student in n time periods during class according to the pre-processed classroom image data; A concentration correction module (30), which is used to correct the concentration of each student in n time periods during class output by the concentration calculation module (20) according to the influence of the concentration between students; The feedback module (40) is used to provide the teacher with feedback on the concentration of each student in n time periods.

2. The teaching interactive system based on artificial intelligence according to claim 1, characterized in that: The image acquisition module (10) comprises a camera unit (101) and an image preprocessing unit (102); The camera unit (101) is constructed based on a camera device and acquires classroom image data in real time; the image preprocessing unit (102) is used to preprocess the classroom image data, including: marking and extracting key frames.

3. The teaching interactive system based on artificial intelligence according to claim 2, characterized in that: The specific steps of the image preprocessing unit (102) preprocessing the classroom image data are as follows: Step S501, uniformly extract m key frames from the classroom image data, where m is a custom parameter; Step S502, using the YOLO algorithm to mark each student in each key frame to generate action image data, each action image data has the same size; Step S503: if the resolution of action image data in a key frame is lower than a preset resolution threshold, the action image data at the same position in the adjacent frame of the key frame in the classroom image data is selected for replacement, and the preset resolution threshold is a custom parameter.

4. The teaching interactive system based on artificial intelligence according to claim 1, characterized in that: The concentration calculation module (20) includes a first hidden layer, a second hidden layer and a classifier; The first hidden layer includes m hidden units; The i-th action image data of one of the students is input into the i-th hidden unit, and the first updated feature is output; All first updated features output by the first hidden layer are input into the second hidden layer, and the second updated features are output; Second, update the feature input classifier and output the student's concentration in n time periods during class; 1≤i≤m, n is a custom parameter.

5. The teaching interactive system based on artificial intelligence according to claim 4 is characterized in that: The m hidden units of the first hidden layer of the concentration calculation module (20) are all constructed based on a convolutional neural network, and the second hidden layer is constructed based on a long short-term memory network.

6. The teaching interactive system based on artificial intelligence according to claim 4, characterized in that: The training samples of the concentration calculation module (20) are as follows: Select L students with similar grades to take a test course at the same time, and obtain classroom image data of L students when taking the test course, and then preprocess them to generate L test action image datasets, each of which contains m key frames, and each test action image dataset is the sample data of a training sample; The test course is divided into n unrelated contents, which are played in sequence, and n test courses and n time periods are set in one-to-one correspondence; After the test course, test questions are given according to the n contents of the test course. The score of each student for each content is the sample label of a training sample. L is a custom parameter.

7. The teaching interactive system based on artificial intelligence according to claim 1, characterized in that: The calculation method of the concentration correction module (30) correcting the output of the concentration calculation module (20) is specifically as follows: ； In the formula, represents the concentration of a student in a certain period of time output by the concentration calculation module (20); It represents the corrected concentration of the student in this period; IF represents the concentration influence coefficient of the student with the lowest concentration in the classroom; d represents the distance between the student and the student with the lowest concentration in the classroom; LPDF represents the difference between the concentration of the student with the lowest concentration in the classroom and the student's own concentration in the previous key frame.

8. The teaching interactive system based on artificial intelligence according to claim 7, characterized in that: The calculation formula of IF is as follows: ； In the formula, BaseInfluence represents the average influence constant of the least focused students in the classroom; HistoricalBehavior represents the historical behavior factor of the least focused students in the classroom; and Represents the first and second weight parameters; BaseInfluence, HistoricalBehavior, and These are custom parameters.

9. The teaching interactive system based on artificial intelligence according to claim 1, characterized in that: The feedback module (40) generates a heat map according to the concentration of each student in each time period, and transmits it to the teacher's client in real time.

10. The teaching interactive system based on artificial intelligence according to claim 5, characterized in that: The calculation formula of the second hidden layer of the concentration calculation module (20) includes: ； ； ； ； ； ； In the formula, , , , , and They respectively represent the input gate, forget gate, output gate, candidate memory state, memory state and hidden state of the second hidden layer of the concentration calculation module (20); wherein, , and Respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to an input gate of a second hidden layer of a concentration calculation module (20); , and Respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to a forget gate of a second hidden layer of a concentration calculation module (20); , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to an output gate of a second hidden layer of a concentration calculation module (20); , and respectively represent a first weight parameter, a second weight parameter and a bias parameter corresponding to the candidate memory state of the second hidden layer of the concentration calculation module (20); represents the sigmoid function; tanh represents the hyperbolic tangent function; Represents the dot product operation; represents the input of the second hidden layer of the concentration calculation module (20); Represents the previous hidden state.

Images