An educational tablet intelligent interaction method and system based on artificial intelligence

Abstract

The present invention relates to the field of educational technology, and in particular to an intelligent interaction method and system for an educational tablet based on artificial intelligence. The method comprises the following steps: the educational tablet uses a classroom behavior recognition model based on a convolutional neural network and a cluster mining algorithm to calculate a student's classroom performance evaluation matrix for classroom images; obtains the proportion of classroom subjects and classroom learning methods through interaction with the teacher and generates a weight matrix; generates a student classroom habit type using the classroom performance evaluation matrix and the weight matrix through a fuzzy comprehensive evaluation method; generates a recommended student seating chart based on the student's classroom habit type, the pre-classroom type set by the teacher's interaction, and the classroom seat template, and displays the recommended student seating chart. The present invention can reduce the influence of classroom subject factors on student classroom performance and restore student learning habits through interaction between the teacher and the educational tablet, so as to generate a more reasonable seating chart based on the student's learning habits.

Description

Technical Field

[0001] The present invention relates to the field of educational technology, and in particular to an educational tablet intelligent interaction method and system based on artificial intelligence. Background Art

[0002] Education, simply put, is about developing good habits. From birth, people begin to understand the world through learning. Learning, as a means of acquiring knowledge and communicating, has become an integral part of daily life. As people interact with the world through learning, their thinking and living habits are deeply influenced by the world, forming personalized learning habits. Therefore, in the information age, with the development and application of new technologies such as big data and artificial intelligence, the education system needs to develop large-scale, personalized teaching models that align with students' learning habits. This is necessary to guide the transformation of traditional education towards "Internet + Education" in the information age.

[0003] Currently, classroom seating plans often fail to fully consider students' learning habits. This results in students with widely varying learning habits being seated close together, leading to interference, influence, and even conflict in class, preventing both students from learning according to their own habits. This also hinders teachers' management and guidance of large student populations. Therefore, in the implementation of "Internet + Education," AI-based models are needed to assist teachers in developing more reasonable seating plans, avoiding inconsistencies caused by teachers' inadequate considerations.

[0004] Therefore, how to enable teachers to interact intelligently with educational tablets to generate seating charts that conform to students' personalized learning habits and class types is a technical problem that needs to be solved. Summary of the Invention

[0005] To this end, the present invention provides an intelligent interactive method and system for an educational tablet based on artificial intelligence, which accurately identifies students' classroom performance through images captured by a camera, and uses a fuzzy comprehensive evaluation method to interactively input a student classroom habit model between the educational tablet and the teacher, thereby reducing the influence of classroom subjects and classroom learning methods on the judgment of students' learning habits based on their classroom performance. This can then identify more credible student classroom habit types and generate a more credible recommended student seating chart to better reduce the mutual interference among students with different learning habits.

[0006] To achieve the above objectives, the present invention proposes an artificial intelligence-based educational tablet intelligent interaction method, comprising:

[0007] Obtain classroom images captured by a camera, and generate a learning record sentence library based on the classroom image through a classroom behavior recognition model based on a convolutional neural network;

[0008] Calculating the student's classroom performance evaluation matrix by using a clustering mining algorithm on the multiple learning record sentence libraries of multiple classes;

[0009] Obtaining the proportion of classroom subjects and classroom learning methods for completed classes through interaction with the teacher, and generating a weight matrix according to the proportion of classroom subjects and classroom learning methods;

[0010] The classroom performance evaluation matrix and the weight matrix are used to generate student classroom habit types through a student classroom habit model based on a fuzzy comprehensive evaluation method;

[0011] A recommended student seating chart is generated based on the students' classroom habit types, the pre-class classroom types interactively set by the teacher, and the classroom seat template, and the recommended student seating chart is displayed.

[0012] Furthermore, the weight matrix includes a primary weight matrix and a secondary weight matrix;

[0013] The first-level indicators of the classroom performance evaluation matrix and the first-level weight matrix include reading and writing performance, class observation learning performance, question-and-answer learning performance, communication learning performance, and distraction performance;

[0014] The secondary indicators of the classroom performance evaluation matrix and the secondary weight matrix are subordinate to the primary indicators, and the secondary indicators include: the reading frequency evaluation level, the writing frequency evaluation level and the reading, reading and writing status evaluation level belonging to the reading and writing performance, the sitting head-up listening evaluation level and the standing listening evaluation level belonging to the watching class learning performance, the hand-raising frequency evaluation level and the standing answering frequency evaluation level belonging to the question-and-answer learning performance, the sitting head-up speaking frequency evaluation level and the sitting head-turning speaking frequency evaluation level belonging to the communication learning performance, and the sitting head-turning listening frequency evaluation level and the head-down sleeping frequency evaluation level belonging to the distraction performance.

[0015] Furthermore, the classroom subjects include liberal arts subjects and science subjects, and the classroom learning method ratio includes the self-study time ratio, lecture time ratio, question-and-answer time ratio, and discussion time ratio;

[0016] The process of generating a weight matrix based on the proportion of the classroom subjects and the classroom learning methods is as follows:

[0017] When the class subject is a liberal arts subject, a first pre-secondary weight matrix is ​​generated; when the class subject is a science subject, a second pre-secondary weight matrix is ​​generated;

[0018] Calculate the adjustment coefficient of the corresponding secondary indicator by adjusting the proportion of the classroom learning method through an adjustment formula;

[0019] Generate the secondary weight matrix by calculation according to the adjustment coefficient and the first pre-secondary weight matrix or the second pre-secondary weight matrix;

[0020] The first-level weight matrix is ​​generated by calculating the proportion of the classroom learning methods.

[0021] Furthermore, the process of generating the student classroom habit type by using the classroom performance evaluation matrix and the weight matrix through the student classroom habit model based on the fuzzy comprehensive evaluation method is as follows:

[0022] The student classroom habit model calculates the first-level score of each first-level indicator according to the classroom performance evaluation matrix and the first-level weight matrix;

[0023] The student classroom habit model determines the first-level indicator with the largest first-level score as the student classroom habit type;

[0024] The student classroom habit model calculates a comprehensive score of the student classroom habits based on the first-level score and the second-level weight matrix.

[0025] Furthermore, the learning record statement library is a statement library based on the xAPI format, and the events of the learning record statement library are associated events of the secondary indicators.

[0026] Furthermore, the process of calculating the student's classroom performance evaluation matrix by using a clustering mining algorithm for the multiple learning record sentence libraries of multiple classes is as follows:

[0027] The associated words of the secondary indicators are retrieved from the learning record sentence library based on the xAPI format, and the associated words are pre-processed and then subjected to a k-means-based clustering mining algorithm to generate the classroom performance evaluation matrix.

[0028] In the above solution, the learning record statement library in the xAPI format lays the foundation for sharing learning data on multiple platforms. The k-means algorithm is used to cluster and mine various information in the learning record statement library in the xAPI format, so that students' high-frequency performance in the classroom can be identified more accurately.

[0029] Furthermore, the classroom behavior recognition model is a three-dimensional convolutional neural network model based on the ResNet101 architecture with a fusion attention mechanism, which sets a self-attention mechanism between the convolution layer and the global pooling layer;

[0030] The process of generating a student's learning record sentence library from the classroom image through a classroom behavior recognition model based on a convolutional neural network is as follows:

[0031] The classroom image is used to generate students' identification features and action state features through the classroom behavior recognition model;

[0032] Matching the action state features with the identification features and storing them to generate a student classroom performance database;

[0033] The learning record sentence library is generated when the duration of the action state feature in the student classroom performance database exceeds a set value.

[0034] Furthermore, the classroom behavior recognition model includes a convolutional regression tracker;

[0035] The convolutional regression tracker tracks and identifies students in the classroom image based on a student classroom performance database.

[0036] In the above solution, the self-attention mechanism is used to reconstruct the feature map in the ResNet101 network to enhance the robustness of the model, and the convolutional regression tracker is used to track and identify students with the same characteristics, thereby achieving accurate identification and monitoring of students' classroom performance.

[0037] Furthermore, the process of generating a recommended student seating chart based on the student's classroom habit type, the pre-class classroom type interactively set by the teacher, and the classroom seat template is as follows:

[0038] For the classroom habit type with the largest number of students, a surrounding seating arrangement is performed in the classroom seat template to generate a surrounding seating block;

[0039] The positions of the surrounding seating blocks are set according to the class type, and the unassigned students are positioned in sequence according to their proximity to the class habit type with the largest number of students to generate the recommended student seating table.

[0040] The present invention also provides a system for applying an artificial intelligence-based educational tablet intelligent interaction method, comprising: an educational tablet and a classroom camera, wherein the educational tablet is equipped with a learning record sentence library generation module, a classroom performance evaluation matrix generation module, a weight matrix generation module, a classroom habit type generation module, and a recommended student seating chart generation module;

[0041] The learning record sentence library generation module is used to obtain classroom images captured by the camera and generate a learning record sentence library based on the classroom image through a classroom behavior recognition model based on a convolutional neural network;

[0042] The classroom performance evaluation matrix generation module is used to calculate the student's classroom performance evaluation matrix by using a clustering mining algorithm on the multiple learning record sentence libraries of multiple classes;

[0043] The weight matrix generation module is used to obtain the proportion of classroom subjects and classroom learning methods for completed classes through interaction with the teacher, and generate a weight matrix based on the classroom subjects and the proportion of classroom learning methods;

[0044] The classroom habit type generation module is used to generate the student's classroom habit type by using the classroom performance evaluation matrix and the weight matrix through a student classroom habit model based on a fuzzy comprehensive evaluation method;

[0045] The recommended student seating chart generation module is used to generate a recommended student seating chart based on the students' classroom habit type, the pre-class classroom type set by the teacher's interaction, and the classroom seat template, and display the recommended student seating chart.

[0046] Compared with the prior art, the present invention has the following advantages:

[0047] 1. Accurately identify students' classroom performance through images collected by the camera, and use the fuzzy comprehensive evaluation method to interactively input students' classroom habits models through the educational tablet and teachers. This can reduce the impact of classroom subjects and classroom learning methods on judging students' learning habits based on their classroom performance, and then identify more credible student classroom habit types and generate more credible recommended student seating charts to better reduce the interference between students with different learning habits.

[0048] 2. The xAPI format learning record statement library lays the foundation for sharing learning data on multiple platforms. The k-means algorithm is used to cluster and mine various information in the xAPI format learning record statement library, making it more accurate to identify students' high-frequency performance in class.

[0049] 3. The self-attention mechanism is used to reconstruct the feature map in the ResNet101 network to enhance the robustness of the model, and the convolutional regression tracker is used to track and identify students with the same characteristics, thereby achieving accurate identification and monitoring of students' classroom performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the general flow of an intelligent interactive method for an educational tablet based on artificial intelligence according to an embodiment of the present invention;

[0051] Figure 2 A detailed flowchart of the fuzzy comprehensive evaluation method of the artificial intelligence-based educational tablet intelligent interaction method according to an embodiment of the present invention;

[0052] Figure 3 This is a structural diagram of a classroom behavior recognition model for an intelligent interactive method for an educational tablet based on artificial intelligence according to an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of the general structure of an artificial intelligence-based educational tablet intelligent interactive system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0056] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside", and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0057] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0058] like Figures 1 to 4 As shown, the present invention provides an intelligent interactive method and system for an educational tablet based on artificial intelligence, which accurately identifies students' classroom performance through images captured by a camera, and uses a student classroom habit model based on a fuzzy comprehensive evaluation method input through interaction between the educational tablet and the teacher to reduce the impact of classroom subjects and classroom learning methods on students' classroom performance, thereby identifying more credible student classroom habit types and generating a more credible recommended student seating chart to better reduce the interference between students with different learning habits.

[0059] like Figures 1 to 4 As shown, this embodiment proposes an artificial intelligence-based educational tablet intelligent interaction method and system. The educational tablet intelligent interaction system applies the artificial intelligence-based educational tablet intelligent interaction method. The educational tablet communicates with the classroom camera. The educational tablet intelligent interaction method includes: the educational tablet obtains the classroom image captured by the camera, and generates a learning record sentence library from the classroom image through a classroom behavior recognition model based on a convolutional neural network;

[0060] The educational tablet calculates the student's classroom performance evaluation matrix using a clustering mining algorithm for the multiple learning record sentence libraries of multiple classes;

[0061] The educational tablet obtains the proportion of classroom subjects and classroom learning methods for completed classes through interaction with the teacher, and generates a weight matrix according to the proportion of classroom subjects and classroom learning methods;

[0062] The educational tablet generates a student classroom habit type by using the classroom performance evaluation matrix and the weight matrix through a student classroom habit model based on a fuzzy comprehensive evaluation method;

[0063] The educational tablet generates a recommended student seating chart based on the student's classroom habit type, the pre-class classroom type interactively set by the teacher, and the classroom seat template, and displays the recommended student seating chart.

[0064] It should be noted that the cameras are one or more cameras set up in the classroom facing the students.

[0065] Furthermore, the structure of the student classroom habit model based on the fuzzy comprehensive evaluation method is shown in the following table, where the element values ​​of the classroom performance evaluation matrix, the first-level weight matrix, and the second-level weight matrix are only for illustration:

[0066]

[0067] In the table, the elements of the first-level weight matrix correspond to the first-level indicators, the elements of the second-level weight matrix correspond to the second-level indicators, and the classroom performance evaluation matrix is ​​the second-level indicator. In the evaluation set V = {V1, V2, V3} = {25, 50, 75}, it corresponds to low, medium, and high evaluation values, among which 25, 50, and 75 constitute the score matrix of the evaluation set.

[0068] According to the above table, it can be concluded that: the weight matrix includes a first-level weight matrix and a second-level weight matrix; the first-level indicators of the classroom performance evaluation matrix and the first-level weight matrix include reading and writing performance, class watching performance, question-and-answer learning performance, communication learning performance and distraction performance; the second-level indicators of the classroom performance evaluation matrix and the second-level weight matrix are subordinate to the first-level indicators, and the second-level indicators include: the reading frequency evaluation level, the writing frequency evaluation level and the reading, reading and writing status evaluation level subordinate to the reading and writing performance, the sitting head-up listening evaluation level and the standing listening evaluation level subordinate to the class watching performance, the hand-raising frequency evaluation level and the standing answering frequency evaluation level subordinate to the question-and-answer learning performance, the sitting head-up speaking frequency evaluation level and the sitting head-turning speaking frequency evaluation level subordinate to the communication learning performance, and the sitting head-turning listening frequency evaluation level and the head-down sleeping frequency evaluation level subordinate to the distraction performance.

[0069] Understandably, different secondary indicators contribute differently to primary indicators. For example, in reading and writing, a high writing frequency is more likely to indicate that students tend to study independently, followed by a high reading frequency, and the lowest level is reading, reading, and writing. In class observation, standing and listening is more likely to indicate that students tend to listen attentively to the teacher's lectures or presentations than sitting and looking up. Therefore, by setting secondary and primary indicators, we can evaluate students' learning habits from multiple dimensions and in a more detailed manner.

[0070] Further, if Figure 1 and 2 As shown, the process of the educational tablet using the classroom performance evaluation matrix and the weight matrix to generate the student classroom habit type through the student classroom habit model based on the fuzzy comprehensive evaluation method is as follows: the student classroom habit model calculates the first-level score of each first-level indicator according to the classroom performance evaluation matrix and the first-level weight matrix; the student classroom habit model determines the first-level indicator with the largest first-level score as the student classroom habit type; the student classroom habit model calculates the student classroom habit comprehensive score according to the first-level score and the second-level weight matrix.

[0071] Specifically, the classroom performance evaluation matrix is ​​the membership matrix of the fuzzy comprehensive evaluation method. The secondary weight matrix and the membership matrix belonging to the same primary indicator are cross-multiplied to obtain a fuzzy evaluation set. The fuzzy evaluation set is cross-multiplied with the score matrix of the evaluation set to obtain the primary score. The primary scores of all primary indicators are generated into an evaluation matrix. The first-level weight vector is cross-multiplied with the evaluation matrix and the score matrix to obtain a comprehensive score of students' classroom habits.

[0072] Further, if Figure 1 and 2 As shown, the classroom subjects include liberal arts subjects and science subjects, and the classroom learning mode ratio includes the self-study time ratio, lecture time ratio, question-and-answer time ratio, and discussion time ratio corresponding to the first four items of the first-level indicators;

[0073] The process of generating a weight matrix according to the classroom subject and the proportion of the classroom learning mode by the educational tablet is as follows: when the classroom subject is a liberal arts subject, generating a first pre-secondary weight matrix; when the classroom subject is a science subject, generating a second pre-secondary weight matrix;

[0074] The proportion of the classroom learning method is calculated through an adjustment formula to obtain the adjustment coefficient of the corresponding secondary indicator; the secondary weight matrix is ​​generated based on the adjustment coefficient and the first pre-secondary weight matrix or the second pre-secondary weight matrix; the first-level weight matrix is ​​generated based on the proportion of the classroom learning method.

[0075] Specifically, the first pre-secondary weight matrix assigns the heaviest weights to reading and writing. This is because liberal arts subjects involve a lot of discussion and teacher explanations. Students' participation in discussions or listening to teachers' lectures is more likely driven by the requirements of the liberal arts subjects rather than their learning habits. This gives the highest weights to reading and writing, which can amplify the influence of students' reading and writing performance on learning habits, making the assessment of learning habits more reliable. For example, the first pre-secondary weight matrix for reading and writing performance is |0.3, 0.6, 0.1| T , the first-pre-secondary weight matrix corresponding to the learning performance of watching lessons is |0.5, 0.5| T .

[0076] Specifically, the weight of the evaluation level of hand-raising frequency in the second pre-secondary weight matrix is ​​greater than the weight of standing answer frequency. For example, the first pre-secondary weight matrix corresponding to question-answering learning performance is |0.8, 0.2| T This is because science subjects are more inclined to students' independent thinking, so it is necessary to increase the frequency of raising hands to judge learning habits.

[0077] More specifically, the elements in the first and second pre-secondary weight matrices of lecture-watching learning performance, question-answering learning performance, communication learning performance, and distraction performance are the same because the impact of science and liberal arts on classroom performance is not much different.

[0078] Specifically, the adjustment formula is a formula for the secondary indicators that are different for liberal arts subjects and science subjects, and is as follows:

[0079]

[0080] In the formula, k1 and k2 represent any group of adjustment coefficients for the reading frequency evaluation level and the writing frequency evaluation level, the sitting head-up listening evaluation level and the standing listening evaluation level, the hand-raising frequency evaluation level and the standing answering frequency evaluation level, respectively; C1, C2, C3 and C4 represent the proportion of self-study time, the proportion of lecture time, the proportion of question-and-answer time and the proportion of discussion time, respectively.

[0081] Preferably, after the adjustment formula, k1 and k2 are normalized and calculated, and the secondary weight matrix is ​​generated according to the product of the adjustment coefficient and the corresponding element in the first pre-secondary weight matrix or the second pre-secondary weight matrix.

[0082] Therefore, the formula was adjusted to make the secondary indicators that differentiate between liberal arts subjects and science subjects more consistent with the actual situation of the curriculum.

[0083] Specifically, the first-level weight matrix is ​​set to 0.5, 0.2, 0.1, and 0.1, respectively, based on the proportion of classroom learning methods. The weight of distraction is fixed at 0.1. For example, if the largest proportion of classroom learning methods is the proportion of self-study time, the corresponding reading and writing performance value in the first-level weight matrix is ​​0.5.

[0084] Furthermore, the learning record statement library is a statement library based on the xAPI format, and the events of the learning record statement library are associated events of the secondary indicators.

[0085] Furthermore, the process of calculating the student's classroom performance evaluation matrix by the educational tablet using a clustering mining algorithm for the multiple learning record sentence libraries of multiple classes is as follows:

[0086] The associated words of the secondary indicators are retrieved from the learning record sentence library based on the xAPI format, and the associated words are pre-processed and then subjected to a k-means-based clustering mining algorithm to generate the classroom performance evaluation matrix.

[0087] In the above solution, the learning record statement library in the xAPI format lays the foundation for sharing learning data on multiple platforms. The k-means algorithm is used to cluster and mine various information in the learning record statement library in the xAPI format, so that students' high-frequency performance in the classroom can be identified more accurately.

[0088] Specifically, the data is structured and standardized through the learning record statement library in the xAPI format, thereby making the data universally applicable to multiple platforms. The learning record statement library in the xAPI format includes time elements, location elements, device elements and events, which facilitates the application of clustering mining algorithms to the multi-granularity data model of the learning record statement library.

[0089] Specifically, the process of performing k-means clustering on the xAPI learning record statement library is as follows: Step S1, data extraction: Retrieving xAPI statements from the learning record store (LRS) of the xAPI learning record statement library. The xAPI statements include the associated words of specific secondary indicators, timestamps, and actor IDs. For example, the associated words for the reading frequency evaluation level are "lower head" and "reading". The data format of the xAPI statements is converted into a Pandas DataFrame data structure; Step S2, data preprocessing: Cleaning the Pandas DataFrame data structure, removing missing values ​​or outliers, and converting it into a numerical form using one-hot encoding; Step S3, performing k-means clustering mining: Using the Scikit-learn machine learning library to implement the k-means clustering algorithm, where the number of clusters k is the number of secondary indicators, the initial cluster center is the characteristic word of each secondary indicator, for example, the characteristic words for the reading frequency evaluation level are "not looking at the podium" and "book open", training a k-means model, and assigning the numerical Pandas DataFrame data structure to each cluster. Therefore, the xAPI learning record statement library is transformed into secondary indicators through k-means clustering.

[0090] Further, see Figure 4 The classroom behavior recognition model is a three-dimensional convolutional neural network model based on the ResNet101 architecture with a fusion attention mechanism, which sets a self-attention mechanism between the convolution layer and the global pooling layer; the process of generating a student's learning record sentence library from the classroom image through the classroom behavior recognition model based on the convolutional neural network is as follows: the classroom image is passed through the classroom behavior recognition model to generate the student's identification features and action state features; the action state features are matched with the identification features and stored to generate a student classroom performance database; the learning record sentence library is generated if the duration of the action state features in the student classroom performance database exceeds the set value.

[0091] It is understandable that the self-attention mechanism can focus on important features to improve model performance. Therefore, the self-attention mechanism is directly added between the convolution layer and the global pooling layer. The self-attention mechanism is used to perform a conventional weighted summation on important features, so that the model can more effectively utilize the local features and correlation information of the input data. Compared with the standard attention calculation module added to the convolutional neural network (CNN), there is no need to transform the data between three-dimensional structures and two-dimensional structures, nor is there any need to add position encoding to mark position information.

[0092] Specifically, the formula of the self-attention mechanism is expressed as:

[0093]

[0094] Where Attention represents the output of the self-attention mechanism, softmax represents the Softmax activation function, Q represents the query of the self-attention mechanism, K represents the key of the self-attention mechanism, and d k represents the dimension of the key, and V represents the value of the self-attention mechanism.

[0095] Specifically, see Figure 4 The ResNet101 architecture includes a 7x7 convolution, an input layer with a stride of 2, and four stages: the first stage consists of 3 Bottleneck residual blocks, the second stage consists of 4 Bottleneck residual blocks, the third stage consists of 23 Bottleneck residual blocks, and the fourth stage consists of 3 Bottleneck residual blocks. The number of output channels and stride of the layers corresponding to the stages increase with the depth of the network. The output layer is a global average pooling layer that compresses the spatial dimension of the feature map to 1x1, and then classifies it through a fully connected layer.

[0096] Specifically, the three-dimensional convolutional neural network model uses samples from the CIFAR-100 dataset. Since the data is not large and is prone to overfitting, data augmentation processing is performed on the CIFAR-100 dataset samples to increase the data volume by more than 8 times. Specifically, the dataset image size is reset to 224×224, and then random cropping, flipping, center cropping and normalization are performed to increase the data volume.

[0097] Further, see Figure 4 The classroom behavior recognition model includes a convolutional regression tracker; the convolutional regression tracker tracks and identifies students in the classroom image based on a student classroom performance database.

[0098] Specifically, see Figure 4The convolutional regression tracker is preferably a scale-adaptive convolutional regression tracker. The scale-adaptive convolutional regression tracker extracts scale-adaptive features for tracking by reusing the output features from the shared backbone network, namely the ResNet101 architecture that integrates the attention mechanism, thereby enhancing the detector in a plug-and-play manner and realizing target tracking detection. Specifically, based on the bounding box to be tracked, the scale-adaptive convolutional regression tracker extracts regional features from two branches of the Siamese network: the RoI of the first branch marks the bounding box range of the object. The width and height of the RoI bounding box of the second branch are expanded 3 times, and its center point and aspect ratio are fixed, marking the region of interest of the search target in the second frame. The scale-adaptive tracker uses a deep convolution operation to calculate the feature correlation between the two feature blocks from the two branches of the Siamese network. Convolutional Regression Network (CRN) uses ridge regression as a layer in a convolutional network for end-to-end training. The network is trained specifically for target tracking tasks, and the obtained feature expression capabilities are stronger. It is understandable that Figure 4 The SVM classifier in is a conventional setting for scale-adaptive convolutional regression trackers.

[0099] In the above solution, the self-attention mechanism is used to reconstruct the feature map in the ResNet101 network to enhance the robustness of the model, and the convolutional regression tracker is used to track and identify students with the same characteristics, thereby achieving accurate identification and monitoring of students' classroom performance.

[0100] Furthermore, the process of generating a recommended student seating chart based on the student's classroom habit type, the pre-class classroom type interactively set by the teacher, and the classroom seating template is as follows:

[0101] The educational tablet arranges surrounding seats for the classroom habit type with the largest number of students in the classroom seating template to generate surrounding seating blocks; the educational tablet sets the positions of the surrounding seating blocks according to the classroom type, and sets the positions of the unassigned students in order of their proximity to the classroom habit type with the largest number of students to generate the recommended student seating table.

[0102] Specifically, the surrounding seating block is centered on the student with the highest comprehensive score for the student's classroom habits. If there are multiple students with the highest comprehensive scores for the student's classroom habits, one is randomly selected. Then, surrounding the student, students are arranged in descending order of the comprehensive scores of the student's classroom habits in the order of left, right, front, back, left front, right front, left back, and right back until all students of the student's classroom habit type are arranged to form a surrounding seating block. For example, if the class type is discussion-based, the order of the student's classroom habit types is communication learning performance, question-and-answer learning performance, class-watching learning performance, reading and writing performance, and distracted performance. That is, if the surrounding seating block is for communication learning performance, it is arranged at the front. Otherwise, it is arranged in the middle front, middle, middle back, or back. For example, if the question-and-answer learning performance is closest to the communication learning performance, then after the surrounding seating block for the question-and-answer learning performance is set, students with question-and-answer learning performance, class-watching learning performance, reading and writing performance, and distracted performance are arranged close to it. For example, if the class type is self-study and question-solving, the order of students' classroom habit types is reading and writing performance, watching class learning performance, question-and-answer learning performance, communication learning performance and distraction performance, and the corresponding seating setting process is the same as above.

[0103] This embodiment also provides a system for applying an artificial intelligence-based educational tablet intelligent interaction method, see Figure 3 The educational tablet communicates with the camera in the classroom. The educational tablet is equipped with a learning record sentence library generation module, a classroom performance evaluation matrix generation module, a weight matrix generation module, a classroom habit type generation module, and a recommended student seating chart generation module;

[0104] The learning record sentence library generation module is used to obtain classroom images captured by the camera and generate a learning record sentence library based on the classroom image through a classroom behavior recognition model based on a convolutional neural network;

[0105] The classroom performance evaluation matrix generation module is used to calculate the student's classroom performance evaluation matrix by using a clustering mining algorithm on the multiple learning record sentence libraries of multiple classes;

[0106] The weight matrix generation module is used to obtain the proportion of classroom subjects and classroom learning methods for completed classes through interaction with the teacher, and generate a weight matrix based on the classroom subjects and the proportion of classroom learning methods;

[0107] The classroom habit type generation module is used to generate the student's classroom habit type by using the classroom performance evaluation matrix and the weight matrix through a student classroom habit model based on a fuzzy comprehensive evaluation method;

[0108] The recommended student seating chart generation module is used to generate a recommended student seating chart based on the students' classroom habit type, the pre-class classroom type set by the teacher's interaction, and the classroom seat template, and display the recommended student seating chart.

[0109] In this embodiment, the images captured by the camera are used to accurately identify students' classroom performance. The student classroom habit model based on the fuzzy comprehensive evaluation method, which is interactively input by the educational tablet and the teacher, can reduce the influence of classroom subjects and classroom learning methods on the judgment of students' learning habits based on their classroom performance. It can then identify more credible types of students' classroom habits and generate more credible recommended student seating charts to better reduce the interference between students with different learning habits. The learning record sentence library in the xAPI format lays the foundation for sharing learning data on multiple platforms. The k-means algorithm is used to cluster and mine various information in the learning record sentence library in the xAPI format, making the identification of students' high-frequency performance in the classroom more accurate. The self-attention mechanism is used to reconstruct the feature map in the ResNet101 network to enhance the robustness of the model. The convolutional regression tracker is used to track and identify students with the same characteristics, achieving accurate identification and monitoring of students' classroom performance.

[0110] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

[0111] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. An educational tablet intelligent interaction method based on artificial intelligence, characterized in that: include: Obtain classroom images captured by a camera, and generate a learning record sentence library based on the classroom image through a classroom behavior recognition model based on a convolutional neural network; Calculating the student's classroom performance evaluation matrix by using a clustering mining algorithm on the multiple learning record sentence libraries of multiple classes; Obtain the teacher's proportion of classroom subjects and classroom learning methods for completed classes through interaction with the educational tablet, and generate a weight matrix based on the proportion of classroom subjects and classroom learning methods; The classroom performance evaluation matrix and the weight matrix are used to generate the student's classroom habit type through a student classroom habit model based on a fuzzy comprehensive evaluation method; generating a recommended student seating chart based on the student's classroom habit type, the pre-class classroom type interactively set by the teacher, and the classroom seating template, and displaying the recommended student seating chart; The weight matrix includes a primary weight matrix and a secondary weight matrix; The first-level indicators of the classroom performance evaluation matrix and the first-level weight matrix include reading and writing performance, class observation learning performance, question-and-answer learning performance, communication learning performance, and distraction performance; The secondary indicators of the classroom performance evaluation matrix and the secondary weight matrix are subordinate to the primary indicators, and the secondary indicators include: the reading frequency evaluation level, the writing frequency evaluation level, and the reading, watching, and writing status evaluation level subordinate to the reading and writing performance; the sitting posture with head raised and listening evaluation level and the standing listening evaluation level subordinate to the class-watching learning performance; the hand-raising frequency evaluation level and the standing answering frequency evaluation level subordinate to the question-answering learning performance; the sitting posture with head raised and speaking frequency evaluation level and the sitting posture with head turned and speaking frequency evaluation level subordinate to the communication learning performance; and the sitting posture with head turned and listening frequency evaluation level and the head lowered and sleeping frequency evaluation level subordinate to the distraction performance; The process of generating the student's classroom habit type by using the classroom performance evaluation matrix and the weight matrix through the student classroom habit model based on the fuzzy comprehensive evaluation method is as follows: The student classroom habit model calculates the first-level score of each first-level indicator according to the classroom performance evaluation matrix and the first-level weight matrix; The student classroom habit model determines the first-level indicator with the largest first-level score as the student's classroom habit type; The student classroom habit model calculates a comprehensive score of the student classroom habits based on the first-level score and the second-level weight matrix.

2. The method for intelligent interaction of an educational tablet based on artificial intelligence according to claim 1, characterized in that: The classroom subjects include liberal arts subjects and science subjects, and the classroom learning method ratio includes the ratio of self-study time, the ratio of lecture time, the ratio of question-and-answer time, and the ratio of discussion time; The process of generating a weight matrix based on the proportion of the classroom subjects and the classroom learning methods is as follows: When the class subject is a liberal arts subject, a first pre-secondary weight matrix is ​​generated; when the class subject is a science subject, a second pre-secondary weight matrix is ​​generated; Calculate the adjustment coefficient of the corresponding secondary indicator by adjusting the proportion of the classroom learning method through an adjustment formula; Generate the secondary weight matrix by calculation according to the adjustment coefficient and the first pre-secondary weight matrix or the second pre-secondary weight matrix; The first-level weight matrix is ​​generated by calculating the proportion of the classroom learning methods.

3. The method for intelligent interaction of an educational tablet based on artificial intelligence according to claim 1, characterized in that: The learning record statement library is a statement library based on the xAPI format, and the events of the learning record statement library are associated events of the secondary indicators.

4. The method for intelligent interaction of an educational tablet based on artificial intelligence according to claim 3, characterized in that: The process of calculating the student's classroom performance evaluation matrix by using a clustering mining algorithm on the multiple learning record sentence libraries of multiple classes is as follows: The associated words of the secondary indicators are retrieved from the learning record sentence library based on the xAPI format, and the associated words are pre-processed and then subjected to a k-means-based clustering mining algorithm to generate the classroom performance evaluation matrix.

5. The method for intelligent interaction with an educational tablet based on artificial intelligence according to any one of claims 1 to 4, characterized in that: The classroom behavior recognition model is a three-dimensional convolutional neural network model based on the ResNet101 architecture with a fusion attention mechanism, which sets a self-attention mechanism between the convolution layer and the global pooling layer; The process of generating a student's learning record sentence library from the classroom image through a classroom behavior recognition model based on a convolutional neural network is as follows: The classroom image is used to generate students' identification features and action state features through the classroom behavior recognition model; Matching the action state features with the identification features and storing them to generate a student classroom performance database; The learning record sentence library is generated when the duration of the action state feature in the student classroom performance database exceeds a set value.

6. The method for intelligent interaction of an educational tablet based on artificial intelligence according to claim 5, characterized in that: The classroom behavior recognition model includes a convolutional regression tracker; The convolutional regression tracker tracks and identifies students in the classroom image based on a student classroom performance database.

7. The method for intelligent interaction with an educational tablet based on artificial intelligence according to any one of claims 1 to 4, characterized in that: The process of generating a recommended student seating chart based on the student's classroom habit type, the pre-class classroom type interactively set by the teacher, and the classroom seat template is as follows: For the classroom habit type with the largest number of students, a surrounding seating arrangement is performed in the classroom seat template to generate a surrounding seating block; The positions of the surrounding seating blocks are set according to the class type, and the unassigned students are positioned in sequence according to their proximity to the class habit type with the largest number of students to generate the recommended student seating table.

8. A system using the artificial intelligence-based educational tablet intelligent interaction method according to any one of claims 1 to 7, characterized in that: include: An educational tablet and a classroom camera, wherein the educational tablet is equipped with a learning record sentence library generation module, a classroom performance evaluation matrix generation module, a weight matrix generation module, a classroom habit type generation module, and a recommended student seating chart generation module; The learning record sentence library generation module is used to obtain classroom images captured by the camera and generate a learning record sentence library based on the classroom image through a classroom behavior recognition model based on a convolutional neural network; The classroom performance evaluation matrix generation module is used to calculate the student's classroom performance evaluation matrix by using a clustering mining algorithm on the multiple learning record sentence libraries of multiple classes; The weight matrix generation module is used to obtain the proportion of classroom subjects and classroom learning methods for completed classes through interaction with the teacher, and generate a weight matrix based on the classroom subjects and the proportion of classroom learning methods; The classroom habit type generation module is used to generate the student's classroom habit type by using the classroom performance evaluation matrix and the weight matrix through a student classroom habit model based on a fuzzy comprehensive evaluation method; The recommended student seating chart generation module is used to generate a recommended student seating chart based on the students' classroom habit type, the pre-class classroom type set by the teacher's interaction, and the classroom seat template, and display the recommended student seating chart.

Images