Education tablet intelligent interaction method and system based on artificial intelligence

Abstract

The invention relates to the technical field of education, in particular to an intelligent interaction method and system for an education tablet based on artificial intelligence, and the method comprises the steps: calculating a classroom performance evaluation matrix of students through a classroom behavior recognition model based on a convolutional neural network and a clustering mining algorithm by the education tablet; obtaining the proportions of classroom subjects and classroom learning modes through interaction with teachers, and generating a weight matrix; generating a student classroom habit type from the classroom performance evaluation matrix and the weight matrix through a fuzzy comprehensive evaluation method; and generating a recommended student seating table according to the class habit types of the students, the class type of the pre-class set by the teacher interaction and the classroom seat template, and displaying the recommended student seating table. Through interaction between the teacher and the education tablet, the influence of classroom subject factors on student classroom performance reduction of learning habits of students can be reduced, and a more reasonable seating list is generated according to the learning habits of the students.

Description

Technical Field

[0001] The present invention relates to the field of educational technology, and in particular to an educational tablet intelligent interaction method and system based on artificial intelligence. Background Art

[0002] Education, in simple terms, is about developing good habits. People have been learning about the world since birth. Learning, as a way to acquire knowledge and communicate, has become an important part of people's daily lives. In the process of interacting with the world through learning, people's thinking habits, living habits, etc. are also deeply influenced by the world, thus forming personalized learning habits. Therefore, in the information age with the development and application of new technologies such as big data and artificial intelligence, the education system needs to develop a large-scale, personalized education and teaching model that conforms to students' learning habits, in order to meet the needs of guiding traditional education to transform and develop into "Internet + Education" in the information age.

[0003] At present, the seating arrangement of a class often does not fully consider the students' learning habits, resulting in students with different learning habits sitting close to each other, interfering, affecting or even conflicting with each other in class, making it impossible for both parties to study according to their own learning habits. This is also not conducive to the management and guidance of teachers facing a large number of students. Therefore, in the implementation of "Internet + Education", it is necessary to assist teachers in formulating a more reasonable seating arrangement through an artificial intelligence-based model to avoid unreasonable seating arrangements due to insufficient consideration by teachers.

[0004] Therefore, how to enable teachers to interact intelligently with educational tablets to generate seating charts that conform to students' personalized learning habits and class types is a technical problem that needs to be solved. Summary of the invention

[0005] To this end, the present invention provides an intelligent interactive method and system for an educational tablet based on artificial intelligence, which can accurately identify students' classroom performance through images captured by a camera, and use a fuzzy comprehensive evaluation method to interactively input a student's classroom habit model between the educational tablet and a teacher, thereby reducing the influence of classroom subjects and classroom learning methods on judging students' learning habits based on their classroom performance, thereby identifying more credible student classroom habit types, and generating a more credible recommended student seating table, so as to better reduce the mutual interference among students with different learning habits.

[0006] To achieve the above purpose, the present invention proposes an educational tablet intelligent interaction method based on artificial intelligence, comprising:

[0007] Obtain classroom images captured by a camera, and generate a learning record sentence library from the classroom images through a classroom behavior recognition model based on a convolutional neural network;

[0008] Calculate the student's classroom performance evaluation matrix by using a cluster mining algorithm for the multiple learning record sentence libraries of multiple classes;

[0009] Obtaining the proportion of classroom subjects and classroom learning methods for completed classes through interaction with the teacher, and generating a weight matrix according to the proportion of classroom subjects and classroom learning methods;

[0010] The classroom performance evaluation matrix and the weight matrix are used to generate the student classroom habit type through the student classroom habit model based on the fuzzy comprehensive evaluation method;

[0011] A recommended student seating chart is generated based on the students' classroom habit types, the pre-class classroom types interactively set by the teacher, and the classroom seat template, and the recommended student seating chart is displayed.

[0012] Further, the weight matrix includes a primary weight matrix and a secondary weight matrix;

[0013] The first-level indicators of the classroom performance evaluation matrix and the first-level weight matrix include reading and writing performance, class watching performance, question-and-answer learning performance, communication learning performance, and distraction performance;

[0014] The secondary indicators of the classroom performance evaluation matrix and the secondary weight matrix are subordinate to the primary indicators, and the secondary indicators include: the reading frequency evaluation level, the writing frequency evaluation level and the reading, reading and writing status evaluation level belonging to the reading and writing performance, the sitting head-up listening evaluation level and the standing listening evaluation level belonging to the watching class learning performance, the hand-raising frequency evaluation level and the standing answering frequency evaluation level belonging to the question-and-answer learning performance, the sitting head-up speaking frequency evaluation level and the sitting head-turning speaking frequency evaluation level belonging to the communication learning performance, and the sitting head-turning listening frequency evaluation level and the head-down sleeping frequency evaluation level belonging to the distraction performance.

[0015] Furthermore, the classroom subjects include liberal arts subjects and science subjects, and the classroom learning method ratio includes the self-study time ratio, the lecture time ratio, the question-and-answer time ratio and the discussion time ratio;

[0016] The process of generating a weight matrix according to the proportion of the classroom subjects and the classroom learning methods is as follows:

[0017] When the classroom subject is a liberal arts subject, a first pre-secondary weight matrix is ​​generated; when the classroom subject is a science subject, a second pre-secondary weight matrix is ​​generated;

[0018] Calculate the adjustment coefficient of the corresponding secondary indicator by adjusting the proportion of the classroom learning method through an adjustment formula;

[0019] The secondary weight matrix is ​​generated by calculation according to the adjustment coefficient and the first pre-secondary weight matrix or the second pre-secondary weight matrix;

[0020] The first-level weight matrix is ​​generated by calculating the proportion of the classroom learning methods.

[0021] Furthermore, the process of generating the student classroom habit type by using the classroom performance evaluation matrix and the weight matrix through the student classroom habit model based on the fuzzy comprehensive evaluation method is as follows:

[0022] The student classroom habit model calculates the first-level score of each first-level indicator according to the classroom performance evaluation matrix and the first-level weight matrix;

[0023] The student classroom habit model determines the first-level indicator with the largest first-level score as the student classroom habit type;

[0024] The student classroom habit model calculates the comprehensive score of the student classroom habits according to the primary score and the secondary weight matrix.

[0025] Furthermore, the learning record statement library is a statement library based on the xAPI format, and the events of the learning record statement library are associated events of the secondary indicators.

[0026] Furthermore, the process of calculating the student's classroom performance evaluation matrix by clustering mining algorithm using the multiple learning record sentence libraries of multiple classes is as follows:

[0027] The associated words of the secondary indicators are retrieved in the learning record sentence library based on the xAPI format, and the classroom performance evaluation matrix is ​​generated by performing a k-means-based clustering mining algorithm after preprocessing the associated words.

[0028] In the above scheme, the learning record sentence library in the xAPI format lays the foundation for sharing learning data on multiple platforms, and the k-means algorithm is used to cluster and mine various information of the learning record sentence library in the xAPI format, so that the high-frequency performance of students in the classroom can be identified more accurately.

[0029] Furthermore, the classroom behavior recognition model is a three-dimensional convolutional neural network model based on the ResNet101 architecture with a fusion attention mechanism, which sets a self-attention mechanism between the convolution layer and the global pooling layer;

[0030] The process of generating a student's learning record sentence library from the classroom image through a classroom behavior recognition model based on a convolutional neural network is as follows:

[0031] The classroom image is used to generate identification features and action state features of students through the classroom behavior recognition model;

[0032] Matching the action state feature with the identification feature and storing it to generate a student classroom performance database;

[0033] The learning record sentence library is generated when the duration of the action state feature in the student classroom performance database exceeds the set value.

[0034] Further, the classroom behavior recognition model includes a convolutional regression tracker;

[0035] The convolutional regression tracker tracks and identifies students in the classroom image based on a student classroom performance database.

[0036] In the above scheme, the self-attention mechanism is used to reconstruct the feature map in the ResNet101 network to enhance the robustness of the model, and the convolutional regression tracker is used to track and identify students with the same characteristics, thereby achieving accurate identification and monitoring of students' classroom performance.

[0037] Furthermore, the process of generating a recommended student seating table according to the class habit type of the student, the class type of the pre-class interactively set by the teacher, and the classroom seat template is as follows:

[0038] For the classroom habit type with the largest number of students, a surrounding seating arrangement is performed in the classroom seat template to generate a surrounding seating block;

[0039] The positions of the surrounding seating blocks are set according to the class type, and the unassigned students are positioned in order according to their proximity to the class habit type with the largest number of students to generate the recommended student seating table.

[0040] The present invention also provides a system for applying the intelligent interactive method of an educational tablet based on artificial intelligence, comprising: an educational tablet and a camera in a classroom, wherein the educational tablet is equipped with a learning record sentence library generation module, a classroom performance evaluation matrix generation module, a weight matrix generation module, a classroom habit type generation module, and a recommended student seating table generation module;

[0041] The learning record sentence library generation module is used to obtain the classroom image captured by the camera, and generate a learning record sentence library through the classroom image through a classroom behavior recognition model based on a convolutional neural network;

[0042] The classroom performance evaluation matrix generation module is used to calculate the student's classroom performance evaluation matrix by using a cluster mining algorithm for a plurality of the learning record sentence libraries of a plurality of classes;

[0043] The weight matrix generation module is used to obtain the proportion of classroom subjects and classroom learning methods for completed classes through interaction with the teacher, and generate a weight matrix according to the proportion of classroom subjects and classroom learning methods;

[0044] The classroom habit type generating module is used to generate the classroom habit type of the students by using the classroom performance evaluation matrix and the weight matrix through the student classroom habit model based on the fuzzy comprehensive evaluation method;

[0045] The recommended student seating chart generation module is used to generate a recommended student seating chart based on the students' classroom habit types, the pre-class classroom types interactively set by the teacher, and the classroom seat template, and to display the recommended student seating chart.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. Accurately identify students' classroom performance through images collected by the camera, and use the fuzzy comprehensive evaluation method to interactively input students' classroom habits through the educational tablet. This can reduce the influence of classroom subjects and classroom learning methods on judging students' learning habits based on their classroom performance, and then identify more credible student classroom habit types, and generate a more credible recommended student seating table to better reduce the interference between students with different learning habits.

[0048] 2. The learning record statement library in the xAPI format lays the foundation for sharing learning data on multiple platforms. The k-means algorithm is used to cluster and mine various information in the learning record statement library in the xAPI format, so that students' high-frequency performance in class can be identified more accurately.

[0049] 3. The self-attention mechanism is used to reconstruct the feature map in the ResNet101 network to enhance the robustness of the model. The convolutional regression tracker is used to track and identify students with the same characteristics, thus achieving accurate identification and monitoring of students' classroom performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the general flow of the intelligent interaction method of an educational tablet based on artificial intelligence according to an embodiment of the present invention;

[0051] Figure 2 A detailed flow chart of a fuzzy comprehensive evaluation method of an intelligent interactive method for an educational tablet based on artificial intelligence according to an embodiment of the present invention;

[0052] Figure 3 A structural diagram of a classroom behavior recognition model of an educational tablet intelligent interaction method based on artificial intelligence according to an embodiment of the present invention;

[0053] Figure 4 The schematic diagram is a general structural diagram of an educational tablet intelligent interactive system based on artificial intelligence according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0056] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0057] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0058] like Figures 1 to 4 As shown, the present invention provides an intelligent interactive method and system for an educational tablet based on artificial intelligence, which can accurately identify students' classroom performance through images captured by a camera, and can reduce the impact of classroom subjects and classroom learning methods on students' classroom performance through a student classroom habit model based on a fuzzy comprehensive evaluation method that is interactively input by the educational tablet and the teacher, thereby identifying more credible student classroom habit types and generating a more credible recommended student seating table to better reduce the mutual interference among students with different learning habits.

[0059] like Figures 1 to 4 As shown, this embodiment proposes an educational tablet intelligent interaction method and system based on artificial intelligence. The educational tablet intelligent interaction system applies an educational tablet intelligent interaction method based on artificial intelligence. The educational tablet communicates with a camera in the classroom. The educational tablet intelligent interaction method includes: the educational tablet obtains a classroom image captured by the camera, and generates a learning record sentence library from the classroom image through a classroom behavior recognition model based on a convolutional neural network;

[0060] The educational tablet uses a clustering mining algorithm to calculate the student's classroom performance evaluation matrix using the multiple learning record sentence libraries of multiple classes;

[0061] The educational tablet obtains the proportion of classroom subjects and classroom learning methods for completed classes through interaction with the teacher, and generates a weight matrix according to the proportion of classroom subjects and classroom learning methods;

[0062] The educational tablet generates a student classroom habit type by using the classroom performance evaluation matrix and the weight matrix through a student classroom habit model based on a fuzzy comprehensive evaluation method;

[0063] The educational tablet generates a recommended student seating chart based on the student's classroom habit type, the classroom type of the pre-conducted class interactively set by the teacher, and the classroom seat template, and displays the recommended student seating chart.

[0064] It should be noted that the camera is one or more cameras set up in the classroom facing the students.

[0065] Furthermore, the structure of the student classroom habit model based on the fuzzy comprehensive evaluation method is shown in the following table, where the element values ​​of the classroom performance evaluation matrix, the first-level weight matrix, and the second-level weight matrix are only for illustration:

[0066]

[0067] In the table, the elements of the first-level weight matrix correspond to the first-level indicators, the elements of the second-level weight matrix correspond to the second-level indicators, and the classroom performance evaluation matrix is ​​the second-level indicator, corresponding to low, medium, and high evaluation values ​​in the evaluation set V = {V1, V2, V3} = {25, 50, 75}, among which 25, 50, and 75 constitute the score matrix of the evaluation set.

[0068] According to the above table, it can be concluded that: the weight matrix includes a first-level weight matrix and a second-level weight matrix; the first-level indicators of the classroom performance evaluation matrix and the first-level weight matrix include reading and writing performance, watching class learning performance, question and answer learning performance, communication learning performance and distraction performance; the second-level indicators of the classroom performance evaluation matrix and the second-level weight matrix are subordinate to the first-level indicators, and the second-level indicators include: the reading frequency evaluation level, the writing frequency evaluation level and the reading, watching and writing status evaluation level belonging to the reading and writing performance, the sitting head-up listening evaluation level and the standing listening evaluation level belonging to the watching class learning performance, the hand-raising frequency evaluation level and the standing answering frequency evaluation level belonging to the question and answer learning performance, the sitting head-up speaking frequency evaluation level and the sitting head-turning speaking frequency evaluation level belonging to the communication learning performance, and the sitting head-turning listening frequency evaluation level and the head-down sleeping frequency evaluation level belonging to the distraction performance.

[0069] It is understandable that different secondary indicators contribute differently to the primary indicators. For example, in reading and writing performance, high writing frequency can better reflect that students' learning habits tend to be independent study, followed by high reading frequency, and reading, reading and writing are the lowest. In the performance of watching classes, standing and listening can better reflect that students' learning habits tend to listen carefully to the teacher's lectures or watch the teacher's presentation materials than sitting and listening with their heads up. Therefore, by setting secondary indicators and primary indicators, we can evaluate students' learning habits from multiple dimensions and in a more detailed way.

[0070] Furthermore, if Figure 1 and 2 As shown, the process of the educational tablet generating the student classroom habit type by using the student classroom habit model based on the fuzzy comprehensive evaluation method from the classroom performance evaluation matrix and the weight matrix is ​​as follows: the student classroom habit model calculates the first-level score of each first-level indicator according to the classroom performance evaluation matrix and the first-level weight matrix; the student classroom habit model determines the first-level indicator with the largest first-level score as the student classroom habit type; the student classroom habit model calculates the student classroom habit comprehensive score according to the first-level score and the second-level weight matrix.

[0071] Specifically, the classroom performance evaluation matrix is ​​a membership matrix of the fuzzy comprehensive evaluation method. The secondary weight matrix and the membership matrix belonging to the same primary indicator are cross-multiplied to obtain a fuzzy evaluation set. The fuzzy evaluation set is cross-multiplied with the score matrix of the evaluation set to obtain the primary score. The primary scores of all primary indicators are used to generate an evaluation matrix. The first-level weight vector is cross-multiplied with the evaluation matrix and the score matrix to obtain a comprehensive score of students' classroom habits.

[0072] Furthermore, if Figure 1 and 2 As shown, the classroom subjects include liberal arts subjects and science subjects, and the classroom learning method ratio includes the self-study time ratio, lecture time ratio, question-and-answer time ratio and discussion time ratio corresponding to the first four items of the first-level indicators in sequence;

[0073] The process of generating a weight matrix according to the classroom subject and the proportion of the classroom learning mode by the educational tablet is as follows: when the classroom subject is a liberal arts subject, generating a first pre-secondary weight matrix; when the classroom subject is a science subject, generating a second pre-secondary weight matrix;

[0074] The adjustment coefficient of the corresponding secondary indicator is calculated by adjusting the proportion of the classroom learning method through an adjustment formula; the secondary weight matrix is ​​generated based on the adjustment coefficient and the first pre-secondary weight matrix or the second pre-secondary weight matrix; the first-level weight matrix is ​​generated based on the proportion of the classroom learning method.

[0075] Specifically, the weights for reading and writing are the largest in the first pre-secondary weight matrix, because there are a lot of contents in liberal arts subjects that need to be discussed with each other and listened to by teachers. At this time, students' participation in discussions or listening to teachers' lectures is more likely to be due to the learning requirements of liberal arts subjects rather than learning habits. At this time, the weights for reading and writing are the largest, which can amplify the judgment tendency of students' reading and writing classroom performance for the evaluation of learning habits, thereby making the judgment of learning habits more authentic and reliable. For example, the first pre-secondary weight matrix corresponding to reading and writing performance is |0.3, 0.6, 0.1| T ,The first-pre-secondary weight matrix corresponding to the performance of watching lessons is |0.5, 0.5| T .

[0076] Specifically, the weight of the evaluation level of hand-raising frequency in the second pre-secondary weight matrix is ​​greater than the weight of standing answer frequency. For example, the first pre-secondary weight matrix corresponding to question-answering learning performance is |0.8, 0.2| T This is because science subjects are more inclined to students' independent thinking, so it is necessary to magnify the frequency of raising hands to judge learning habits.

[0077] More specifically, the elements in the first pre-secondary weight matrix and the second pre-secondary weight matrix of lecture watching learning performance, question-and-answer learning performance, communication learning performance, and distraction performance are the same because the impact of science and liberal arts on classroom performance is not much different.

[0078] Specifically, the adjustment formula is a formula for the secondary indicators that are different for liberal arts subjects and science subjects, and is as follows:

[0079]

[0080] In the formula, k1 and k2 represent any group of adjustment coefficients for the reading frequency evaluation level and the writing frequency evaluation level, the sitting head-up listening evaluation level and the standing listening evaluation level, the hand-raising frequency evaluation level and the standing answering frequency evaluation level, respectively. C1, C2, C3 and C4 represent the proportion of self-study time, the proportion of lecture time, the proportion of question and answer time and the proportion of discussion time respectively.

[0081] Preferably, after the adjustment formula, k1 and k2 are normalized and calculated, and the secondary weight matrix is ​​generated according to the product of the adjustment coefficient and the corresponding element in the first pre-secondary weight matrix or the second pre-secondary weight matrix.

[0082] Therefore, adjusting the formula to make the secondary indicators that differentiate between liberal arts subjects and science subjects more in line with the actual situation of the curriculum.

[0083] Specifically, the first-level weight matrix is ​​set to 0.5, 0.2, 0.1, and 0.1 in order according to the proportion of classroom learning methods, and the weight of distraction performance is a fixed value of 0.1. For example, if the classroom learning method with the largest proportion is the proportion of self-study time, the corresponding reading and writing performance in the first-level weight matrix is ​​0.5.

[0084] Furthermore, the learning record statement library is a statement library based on the xAPI format, and the events of the learning record statement library are associated events of the secondary indicators.

[0085] Furthermore, the process of the educational tablet calculating the student's classroom performance evaluation matrix by using a clustering mining algorithm for the multiple learning record sentence libraries of multiple classes is as follows:

[0086] The associated words of the secondary indicators are retrieved in the learning record sentence library based on the xAPI format, and the classroom performance evaluation matrix is ​​generated by performing a k-means-based clustering mining algorithm after preprocessing the associated words.

[0087] In the above scheme, the learning record sentence library in the xAPI format lays the foundation for sharing learning data on multiple platforms, and the k-means algorithm is used to cluster and mine various information of the learning record sentence library in the xAPI format, so that the high-frequency performance of students in the classroom can be identified more accurately.

[0088] Specifically, the data is structured and standardized through the learning record statement library in the xAPI format, so that the data can be used on multiple platforms. The learning record statement library in the xAPI format includes time elements, location elements, device elements and events, which facilitates the use of clustering mining algorithms in the multi-granularity data model of the learning record statement library.

[0089] Specifically, the process of performing k-means clustering on the xAPI learning record statement library is as follows: Step S1, data extraction: retrieve xAPI statements from the learning record store (LRS) of the xAPI learning record statement library, the xAPI statements include the associated words, timestamps and actor IDs of specific secondary indicators, for example, the associated words of the reading frequency evaluation level are bowing and reading, and the data format of the xAPI statements is converted into a Pandas DataFrame data structure; Step S2, data preprocessing: clean the data of the Pandas DataFrame data structure, remove missing values ​​or outliers, and convert it into a numerical form using One-Hot Encoding; Step S3, perform k-means clustering mining: use the Scikit-learn machine learning library to implement the k-means clustering algorithm, where the number of clusters k is the number of secondary indicators, and the initial cluster center is the characteristic word of each secondary indicator, for example, the characteristic words of the reading frequency evaluation level are not looking at the podium and the book is open, train the k-means model, and distribute the numerical Pandas DataFrame data structure to each cluster. Therefore, the xAPI learning record statement library is transformed into secondary indicators through k-means clustering.

[0090] Further, see Figure 4 The classroom behavior recognition model is a three-dimensional convolutional neural network model based on the ResNet101 architecture with a fusion attention mechanism, which sets a self-attention mechanism between the convolution layer and the global pooling layer; the process of generating a student's learning record sentence library from the classroom image through the classroom behavior recognition model based on the convolutional neural network is as follows: the classroom image is passed through the classroom behavior recognition model to generate the student's identification features and action state features; the action state features are matched and stored with the identification features to generate a student classroom performance database; the learning record sentence library is generated if the duration of the action state features in the student classroom performance database exceeds the set value.

[0091] It is understandable that the self-attention mechanism can focus on important features to improve model performance. Therefore, the self-attention mechanism is directly added between the convolution layer and the global pooling layer. The self-attention mechanism is used to perform a conventional weighted summation on important features, so that the model can more effectively utilize the local features and correlation information of the input data. Compared with the standard attention calculation module added to the convolutional neural network (CNN), there is no need to transform the data between the three-dimensional structure and the two-dimensional structure, nor is there any need to add position encoding to mark the position information.

[0092] Specifically, the formula of the self-attention mechanism is expressed as:

[0093]

[0094] Where Attention represents the output of the self-attention mechanism, softmax represents the Softmax activation function, Q represents the query of the self-attention mechanism, K represents the key of the self-attention mechanism, and d k represents the dimension of the key and V represents the value of the self-attention mechanism.

[0095] Specifically, see Figure 4 The ResNet101 architecture includes a 7x7 convolution, an input layer with a stride of 2, and four stages: the first stage consists of 3 Bottleneck residual blocks, the second stage consists of 4 Bottleneck residual blocks, the third stage consists of 23 Bottleneck residual blocks, and the fourth stage consists of 3 Bottleneck residual blocks. The number of output channels and the stride of the layer corresponding to the stage increase with the depth of the network. The output layer is a global average pooling layer that compresses the spatial dimension of the feature map to 1x1, and then classifies it through a fully connected layer.

[0096] Specifically, the three-dimensional convolutional neural network model uses CIFAR-100 dataset samples. Since the data is not large and is prone to overfitting, data enhancement processing is performed on the CIFAR-100 dataset samples to increase the data volume by more than 8 times. Specifically, the dataset image size is reset to 224×224, and then random cropping, flipping, center cropping and normalization are performed to increase the data volume.

[0097] Further, see Figure 4 The classroom behavior recognition model includes a convolutional regression tracker; the convolutional regression tracker tracks and identifies students in the classroom image based on a student classroom performance database.

[0098] Specifically, see Figure 4The convolutional regression tracker is preferably a scale-adaptive convolutional regression tracker, which extracts scale-adaptive features for tracking by reusing the output features from the shared backbone network (backbone), i.e., the ResNet101 architecture with fused attention mechanism, thereby enhancing the detector in a plug-and-play manner and realizing target tracking detection. Specifically, based on the bounding box to be tracked, the scale-adaptive convolutional regression tracker extracts regional features from two branches of the siamese network: the RoI of the first branch marks the bounding box range of the object. The width and height of the RoI bounding box of the second branch are expanded 3 times, with a fixed center point and aspect ratio, marking the region of interest of the search target in the second frame. The scale-adaptive tracker uses deep convolution operations to calculate the feature correlation between two feature blocks from the two branches of the siamese network. Convolutional Regression Network (CRN) uses ridge regression as a layer in a convolutional network for end-to-end training. The network is trained specifically for target tracking tasks, and the obtained features have stronger expressive power. It is understandable that Figure 4 The SVM classifier in is a conventional setting for scale-adaptive convolutional regression trackers.

[0099] In the above scheme, the self-attention mechanism is used to reconstruct the feature map in the ResNet101 network to enhance the robustness of the model, and the convolutional regression tracker is used to track and identify students with the same characteristics, thereby achieving accurate identification and monitoring of students' classroom performance.

[0100] Furthermore, the process of generating a recommended student seating table according to the classroom habit type of the student, the classroom type of the pre-conducted class interactively set by the teacher, and the classroom seat template is as follows:

[0101] The educational tablet arranges surrounding seats in the classroom seat template for the classroom habit type with the largest number of students to generate surrounding seating blocks; the educational tablet sets the positions of the surrounding seating blocks according to the classroom type, and sets the positions of the unarranged students in order according to their proximity to the classroom habit type with the largest number of students to generate the recommended student seating table.

[0102] Specifically, the surrounding seating block takes the student with the highest comprehensive score of the student's classroom habits as the center. If there are multiple students with the highest comprehensive score of the student's classroom habits, one is randomly selected, and then the surrounding seating block is arranged in the order of left, right, front, back, left front, right front, left back, and right back according to the comprehensive score of the student's classroom habits from high to low until all students of the student's classroom habit type are set to form a surrounding seating block. For example, if the class type is discussion type, the order of the student's classroom habit type is communication learning performance, question and answer learning performance, class watching learning performance, reading and writing performance, and distraction performance, that is, if the surrounding seating block is for communication learning performance, it is arranged at the front, otherwise it is arranged in the middle front, middle, middle back or back in sequence. For example, if the question and answer learning performance is the closest to the communication learning performance, then after the surrounding seating block of the question and answer learning performance is set, the students with question and answer learning performance, class watching learning performance, reading and writing performance, and distraction performance are arranged close to it in sequence. For example, if the class type is a self-study and question-solving type, the order of students' classroom habit types is reading and writing performance, watching class learning performance, question-and-answer learning performance, communication learning performance and distraction performance, and the corresponding seating arrangement process is the same as above.

[0103] This embodiment also provides a system for applying an educational tablet intelligent interaction method based on artificial intelligence, see Figure 3 , the educational tablet communicates with the camera of the classroom, and the educational tablet is equipped with a learning record sentence library generation module, a classroom performance evaluation matrix generation module, a weight matrix generation module, a classroom habit type generation module and a recommended student seating table generation module;

[0104] The learning record sentence library generation module is used to obtain the classroom image captured by the camera, and generate a learning record sentence library through the classroom image through a classroom behavior recognition model based on a convolutional neural network;

[0105] The classroom performance evaluation matrix generation module is used to calculate the student's classroom performance evaluation matrix by using a cluster mining algorithm for a plurality of the learning record sentence libraries of a plurality of classes;

[0106] The weight matrix generation module is used to obtain the proportion of classroom subjects and classroom learning methods for completed classes through interaction with the teacher, and generate a weight matrix according to the proportion of classroom subjects and classroom learning methods;

[0107] The classroom habit type generating module is used to generate the classroom habit type of the students by using the classroom performance evaluation matrix and the weight matrix through the student classroom habit model based on the fuzzy comprehensive evaluation method;

[0108] The recommended student seating chart generation module is used to generate a recommended student seating chart based on the students' classroom habit types, the pre-class classroom types interactively set by the teacher, and the classroom seat template, and to display the recommended student seating chart.

[0109] In this embodiment, the classroom performance of students is accurately identified through images collected by the camera, and the classroom habit model of students based on the fuzzy comprehensive evaluation method that is interactively input by the educational tablet and the teacher can reduce the influence of classroom subjects and classroom learning methods on judging students' learning habits based on their classroom performance, and then more credible types of students' classroom habits can be identified, and a more credible recommended student seating table can be generated to better reduce the mutual interference of students with different learning habits. The learning record sentence library in the xAPI format lays the foundation for sharing learning data on multiple platforms, and the k-means algorithm is used to cluster and mine various information in the learning record sentence library in the xAPI format, so that the high-frequency performance of students in the classroom can be more accurately identified. The feature map in the ResNet101 network is reconstructed using the self-attention mechanism to enhance the robustness of the model, and the convolutional regression tracker is used to track and identify students with the same characteristics, so as to achieve accurate recognition and monitoring of students' classroom performance.

[0110] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An educational tablet intelligent interaction method based on artificial intelligence, characterized in that: include: Obtain classroom images captured by a camera, and generate a learning record sentence library using the classroom images through a classroom behavior recognition model based on a convolutional neural network; Calculate the student's classroom performance evaluation matrix by using a cluster mining algorithm for the multiple learning record sentence libraries of multiple classes; Obtaining the proportion of classroom subjects and classroom learning methods for completed classes through interaction with the teacher, and generating a weight matrix according to the proportion of classroom subjects and classroom learning methods; The classroom performance evaluation matrix and the weight matrix are used to generate the student classroom habit type through the student classroom habit model based on the fuzzy comprehensive evaluation method; A recommended student seating chart is generated based on the students' classroom habit types, the pre-class classroom types interactively set by the teacher, and the classroom seat template, and the recommended student seating chart is displayed.

2. The method for intelligent interaction of an educational tablet based on artificial intelligence according to claim 1, characterized in that: The weight matrix includes a primary weight matrix and a secondary weight matrix; The first-level indicators of the classroom performance evaluation matrix and the first-level weight matrix include reading and writing performance, class watching performance, question-and-answer learning performance, communication learning performance, and distraction performance; The secondary indicators of the classroom performance evaluation matrix and the secondary weight matrix are subordinate to the primary indicators, and the secondary indicators include: the reading frequency evaluation level, the writing frequency evaluation level and the reading, reading and writing status evaluation level belonging to the reading and writing performance, the sitting head-up listening evaluation level and the standing listening evaluation level belonging to the watching class learning performance, the hand-raising frequency evaluation level and the standing answering frequency evaluation level belonging to the question-and-answer learning performance, the sitting head-up speaking frequency evaluation level and the sitting head-turning speaking frequency evaluation level belonging to the communication learning performance, and the sitting head-turning listening frequency evaluation level and the head-down sleeping frequency evaluation level belonging to the distraction performance.

3. The method for intelligent interaction of an educational tablet based on artificial intelligence according to claim 2, characterized in that: The classroom subjects include liberal arts subjects and science subjects, and the classroom learning method ratio includes the self-study time ratio, lecture time ratio, question-and-answer time ratio and discussion time ratio; The process of generating a weight matrix according to the proportion of the classroom subjects and the classroom learning methods is as follows: When the classroom subject is a liberal arts subject, a first pre-secondary weight matrix is ​​generated; when the classroom subject is a science subject, a second pre-secondary weight matrix is ​​generated; Calculate the adjustment coefficient of the corresponding secondary indicator by adjusting the proportion of the classroom learning method through an adjustment formula; The secondary weight matrix is ​​generated by calculation according to the adjustment coefficient and the first pre-secondary weight matrix or the second pre-secondary weight matrix; The first-level weight matrix is ​​generated by calculating the proportion of the classroom learning methods.

4. The method for intelligent interaction of an educational tablet based on artificial intelligence according to claim 2, characterized in that: The process of generating the student classroom habit type by using the classroom performance evaluation matrix and the weight matrix through the student classroom habit model based on the fuzzy comprehensive evaluation method is as follows: The student classroom habit model calculates the first-level score of each first-level indicator according to the classroom performance evaluation matrix and the first-level weight matrix; The student classroom habit model determines the first-level indicator with the largest first-level score as the student classroom habit type; The student classroom habit model calculates the comprehensive score of the student classroom habits according to the primary score and the secondary weight matrix.

5. The method for intelligent interaction of an educational tablet based on artificial intelligence according to claim 2, characterized in that: The learning record statement library is a statement library based on the xAPI format, and the events of the learning record statement library are associated events of the secondary indicators.

6. The method for intelligent interaction of an educational tablet based on artificial intelligence according to claim 5, characterized in that: The process of calculating the student's classroom performance evaluation matrix by using a clustering mining algorithm for the multiple learning record sentence libraries of multiple classes is as follows: The associated words of the secondary indicators are retrieved in the learning record sentence library based on the xAPI format, and the classroom performance evaluation matrix is ​​generated by performing a k-means-based clustering mining algorithm after preprocessing the associated words.

7. The method for intelligent interaction of an educational tablet based on artificial intelligence according to any one of claims 1 to 6, characterized in that: The classroom behavior recognition model is a three-dimensional convolutional neural network model based on the ResNet101 architecture with a fusion attention mechanism, which sets a self-attention mechanism between the convolution layer and the global pooling layer; The process of generating a student's learning record sentence library by using the classroom image through a classroom behavior recognition model based on a convolutional neural network is as follows: The classroom image is used to generate students' identification features and action state features through the classroom behavior recognition model; Matching the action state feature with the identification feature and storing it to generate a student classroom performance database; The learning record sentence library is generated when the duration of the action state feature in the student classroom performance database exceeds the set value.

8. The method for intelligent interaction of an educational tablet based on artificial intelligence according to claim 7, characterized in that: The classroom behavior recognition model includes a convolutional regression tracker; The convolutional regression tracker tracks and identifies students in the classroom image based on a student classroom performance database.

9. The method for intelligent interaction of an educational tablet based on artificial intelligence according to any one of claims 1 to 6, characterized in that: The process of generating a recommended student seating table according to the class habit type of the students, the class type of the pre-class interactively set by the teacher, and the classroom seat template is as follows: For the classroom habit type with the largest number of students, a surrounding seating arrangement is performed in the classroom seat template to generate a surrounding seating block; The positions of the surrounding seating blocks are set according to the class type, and the unassigned students are positioned in order according to their proximity to the class habit type with the largest number of students to generate the recommended student seating table.

10. A system using the artificial intelligence-based educational tablet intelligent interaction method according to any one of claims 1 to 9, characterized in that: include: An educational tablet and a camera in the classroom, wherein the educational tablet is equipped with a learning record sentence library generation module, a classroom performance evaluation matrix generation module, a weight matrix generation module, a classroom habit type generation module, and a recommended student seating table generation module; The learning record sentence library generation module is used to obtain the classroom image captured by the camera, and generate a learning record sentence library through the classroom image through a classroom behavior recognition model based on a convolutional neural network; The classroom performance evaluation matrix generation module is used to calculate the student's classroom performance evaluation matrix by using a cluster mining algorithm for a plurality of the learning record sentence libraries of a plurality of classes; The weight matrix generation module is used to obtain the proportion of classroom subjects and classroom learning methods for completed classes through interaction with the teacher, and generate a weight matrix according to the proportion of classroom subjects and classroom learning methods; The classroom habit type generating module is used to generate the classroom habit type of the students by using the classroom performance evaluation matrix and the weight matrix through the student classroom habit model based on the fuzzy comprehensive evaluation method; The recommended student seating chart generation module is used to generate a recommended student seating chart based on the students' classroom habit types, the pre-class classroom types interactively set by the teacher, and the classroom seat template, and to display the recommended student seating chart.

Images