Online education management method and system based on artificial intelligence

Abstract

The invention discloses an online education management method and system based on artificial intelligence, and the method comprises the steps: carrying out the semantic analysis of a student head state monitoring video collected by a camera through employing a computer vision technology based on deep learning, and mining the semantic feature representation of the head state of each key frame in the video; and performing global time sequence behavior pattern analysis on the head state of the student so as to intelligently evaluate whether the student focuses on listening, thereby facilitating implementation of effective teaching management intervention. Therefore, real-time monitoring of the concentration degree of the students in the online education environment can be realized, the perception ability of teachers to the learning states of the students is improved, distraction behaviors of the students are found and corrected in time, and thus the quality and effect of online teaching are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent management, and more specifically, to an online education management method and system based on artificial intelligence. Background Art

[0002] Online education, as a product of the rapid progress of information technology, is gradually becoming an indispensable part of the modern education system. Relying on the network platform, online education has achieved seamless connection between teachers' remote teaching and students' online learning, which has completely subverted the traditional teaching mode in which teachers and students must communicate face to face, greatly broadened the spatial boundaries of teaching and learning, and made knowledge dissemination and learning experience more flexible and convenient.

[0003] However, the unbounded nature of online education also poses a severe challenge to its teaching management. In an online learning environment, students are easily distracted by various non-learning factors such as social media and games, which affects their concentration and leads to a decrease in learning efficiency. At the same time, due to the lack of intuitive face-to-face interaction between teachers and students, it is difficult for teachers to capture students' learning dynamics in a timely manner, such as distraction, confusion, etc., and thus cannot provide timely and effective teaching feedback and guidance, making the quality assurance of online education more difficult than traditional education.

[0004] Therefore, we look forward to an online education management method based on artificial intelligence that can monitor and evaluate students' learning concentration, promptly detect and intervene in students' inattentive behaviors, and improve the teaching quality and effectiveness of online education. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an online education management method and system based on artificial intelligence, which performs semantic analysis on the student head state monitoring video collected by the camera by using computer vision technology based on deep learning, and mines the semantic feature representation of the head state of each key frame in the video, and then analyzes the global temporal behavior pattern of the student's head state, so as to intelligently evaluate whether the student is concentrating on listening, so as to implement effective teaching management intervention. In this way, real-time monitoring of student concentration in an online education environment can be achieved, teachers' perception of students' learning status can be improved, and students' distracting behaviors can be discovered and corrected in time, thereby improving the quality and effectiveness of online teaching.

[0006] According to one aspect of the present application, there is provided an online education management method based on artificial intelligence, which includes: Obtaining a head status monitoring video of a monitored student object collected by a camera; Performing key frame semantic coding on the head state monitoring video to obtain a time series of head state semantic coding feature vectors; Inputting the time series of the head state semantic encoding feature vectors into a semantic association enhancer based on feature energy level measurement association radiation to obtain a time series of enhanced head state semantic encoding feature vectors; Determining whether the monitored student object is focused based on the head state temporal behavior pattern characteristics of the time series of the enhanced head state semantic encoding feature vector; In response to the identification result that the monitored student object is not concentrating, a reminder subtitle is generated to prompt the teacher to pay attention to the learning status of the monitored student object.

[0007] According to another aspect of the present application, there is provided an online education management system based on artificial intelligence, which includes: A monitoring video acquisition module is used to acquire the head status monitoring video of the monitored student object collected by the camera; A key frame semantic coding module, used for performing key frame semantic coding on the head state monitoring video to obtain a time series of head state semantic coding feature vectors; A semantic association enhancement module, used for inputting the time series of the head state semantic encoding feature vector into a semantic association enhancer based on feature energy level measurement association radiation to obtain a time series of enhanced head state semantic encoding feature vector; A concentration state determination module, used to determine whether the monitored student object is concentrating based on the head state temporal behavior pattern characteristics of the time series of the enhanced head state semantic encoding feature vector; The reminder subtitle generation module is used to generate reminder subtitles to prompt the teacher to pay attention to the learning status of the monitored student object in response to the recognition result that the monitored student object is not focused.

[0008] Compared with the prior art, the present application provides an online education management method and system based on artificial intelligence. It uses computer vision technology based on deep learning to perform semantic analysis on the student head status monitoring video collected by the camera, and mines the semantic feature representation of the head status of each key frame in the video, and then analyzes the global temporal behavior pattern of the student's head status, so as to intelligently evaluate whether the student is concentrating on listening, so as to implement effective teaching management intervention. In this way, it is possible to achieve real-time monitoring of student concentration in the online education environment, improve teachers' perception of students' learning status, and promptly discover and correct students' distracted behavior, thereby improving the quality and effectiveness of online teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 It is a flowchart of an online education management method based on artificial intelligence according to an embodiment of the present application; Figure 2 A data flow diagram of an online education management method based on artificial intelligence according to an embodiment of the present application; Figure 3 It is a flowchart of the training phase of the online education management method based on artificial intelligence according to an embodiment of the present application; Figure 4 It is a flowchart of sub-step S2 of the online education management method based on artificial intelligence according to an embodiment of the present application; Figure 5 It is a flowchart of sub-step S4 of the online education management method based on artificial intelligence according to an embodiment of the present application; Figure 6 It is a block diagram of an online education management system based on artificial intelligence according to an embodiment of the present application. DETAILED DESCRIPTION

[0011] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0012] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0013] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0014] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0015] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0016] In an online learning environment, students are easily distracted by various non-learning factors such as social media and games, which affect their concentration and lead to reduced learning efficiency. At the same time, due to the lack of intuitive face-to-face interaction between teachers and students, it is difficult for teachers to capture students' learning dynamics in a timely manner, such as distraction, confusion, etc., and thus cannot provide timely and effective teaching feedback and guidance, making the quality assurance of online education more difficult than traditional education. Therefore, it is expected that an online education management method based on artificial intelligence can monitor and evaluate students' learning concentration, timely detect and intervene in students' non-focused behaviors, so as to improve the teaching quality and effectiveness of online education.

[0017] In the technical solution of the present application, an online education management method based on artificial intelligence is proposed. Figure 1 This is a flowchart of an online education management method based on artificial intelligence according to an embodiment of the present application. Figure 2 : is a data flow diagram of an online education management method based on artificial intelligence according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the online education management method based on artificial intelligence includes the steps of: S1, obtaining a head state monitoring video of a monitored student object captured by a camera; S2, performing key frame semantic encoding on the head state monitoring video to obtain a time series of head state semantic encoding feature vectors; S3, inputting the time series of the head state semantic encoding feature vectors into a semantic association enhancer based on feature energy level metric associated radiation to obtain a time series of enhanced head state semantic encoding feature vectors; S4, determining whether the monitored student object is focused based on the head state temporal behavior pattern characteristics of the time series of the enhanced head state semantic encoding feature vectors; S5, in response to the recognition result that the monitored student object is not focused, generating a reminder subtitle to prompt the teacher to pay attention to the learning status of the monitored student object.

[0018] In particular, the S1 obtains a monitoring video of the head state of the monitored student object collected by the camera. In an embodiment of the present application, the camera is integrated in a student device terminal, such as a front camera of a computer or tablet computer, to ensure that the dynamic state of the student's head can be captured while respecting the student's privacy.

[0019] In particular, the S2 performs key frame semantic coding on the head state monitoring video to obtain a time series of head state semantic coding feature vectors. In particular, in a specific example of the present application, Figure 4 As shown, the S2 includes: S21, sampling the key frames of the head status monitoring video at a predetermined sampling frequency to obtain a time series of head status monitoring key frames; S22, semantically textually expressing each head status monitoring key frame in the time series of the head status monitoring key frames to obtain a time series of head status text descriptions; S23, semantically encoding each head status text description in the time series of the head status text descriptions to obtain a time series of the head status semantic encoding feature vectors.

[0020] Specifically, the S21 performs key frame sampling on the head status monitoring video at a predetermined sampling frequency to obtain a time series of key frames for head status monitoring. Considering that the original video data collected by the camera is usually very large and contains a large amount of redundant information. If the entire video is processed directly, the amount of calculation and processing time will be greatly increased. Therefore, in the technical solution of the present application, the head status monitoring video is processed by key frame sampling. By setting a reasonable predetermined sampling frequency, balancing processing efficiency and information loss, only representative frames in the video are selected for processing, thereby obtaining a time series of key frames for head status monitoring, effectively reducing the complexity of subsequent processing.

[0021] Specifically, in S22, each head state monitoring key frame in the time sequence of the head state monitoring key frame is semantically expressed in text form to obtain a time sequence of the head state text description. In the technical solution of the present application, each head state monitoring key frame in the time sequence of the head state monitoring key frame is input into a head semantic text expressor based on an encoder-decoder structure to obtain a time sequence of the head state text description. Since the original video frame contains a large amount of redundant information (such as background, light changes, etc.), this information is not of substantial help in evaluating the concentration of students, and may even introduce noise, affecting the accuracy of the analysis. Therefore, in order to extract the core information related to the concentration of students, in the technical solution of the present application, a head semantic text expressor based on an encoder-decoder structure is used to express each head state monitoring key frame in the time sequence of the head state monitoring key frame in semantic text form to obtain a time sequence of the head state text description. Through this text expression method, redundant information in the video frame can be effectively removed, only information directly related to the student's head state is retained, and the key movements and expressions of the student's head, such as nodding, shaking head, and sight shifting, are concisely expressed, providing a concise and effective information representation for subsequent concentration evaluation.

[0022] Specifically, the S23 semantically encodes each head state text description in the time series of the head state text description to obtain the time series of the head state semantic encoding feature vector. That is, in the technical solution of the present application, semantically encodes each head state text description in the time series of the head state text description, and digs out the deep semantic features therein to obtain the time series of the head state semantic encoding feature vector. In an embodiment of the present application, a pre-trained deep learning model, such as BERT or Transformer, can be used as a semantic encoder, and its semantic representation ability learned on a large amount of text data is used to capture the key information such as the expression, eyes, head posture, etc. of the students in the head state text description, and convert the text description into a numerical feature vector representation that can be understood by a computer, so as to provide strong feature support for the subsequent concentration judgment.

[0023] It is worth mentioning that in other specific examples of the present application, the head status monitoring video can also be semantically encoded with key frames in other ways to obtain a time series of head status semantic encoding feature vectors, for example: input the head status monitoring video; extract key frames from the head status monitoring video; perform feature extraction on each key frame, and use a pre-trained deep learning model (such as CNN or Transformer) to extract the semantic features of the key frames; encode the semantic features extracted from each key frame into a feature vector; and organize the feature vectors generated by each key frame into a time series in chronological order to obtain a time series of head status semantic encoding feature vectors.

[0024] In particular, S3 inputs the time series of the head state semantic coding feature vector into a semantic association enhancer based on feature energy level measurement associated radiation to obtain a time series of enhanced head state semantic coding feature vectors. Considering that in the head state monitoring video, the importance of head state features at different time points for concentration assessment is different. Therefore, in the technical solution of the present application, in order to improve the expressive power of the time series of the head state semantic coding feature vector, a semantic association enhancer based on feature energy level measurement associated radiation is further introduced to process it to obtain a time series of enhanced head state semantic coding feature vectors. Specifically, the semantic association enhancer quantifies the energy distribution of each head state semantic coding feature vector by calculating its feature energy level coefficient, so as to represent the semantic information content density of each feature vector itself and its influence on the semantic information of other feature vectors in the surrounding neighborhood. Then, through a series of convolution and activation operations, the local dependency association relationship between the feature energy level coefficients of the feature vector is captured, and the attention weight is generated thereby. Furthermore, based on the correlation radiation mechanism, the time series of the head state semantic encoding feature vector is weighted and residual processed to enhance the expression of key features and weaken the influence of irrelevant features, thereby obtaining a time series of enhanced head state semantic encoding feature vectors, which effectively improves the accuracy of subsequent concentration assessment.

[0025] In an embodiment of the present application, the time series of the head state semantic coding feature vector is input into a semantic association enhancer based on feature energy level metric association radiation to obtain a time series of enhanced head state semantic coding feature vectors, including: calculating the feature energy level coefficients of each head state semantic coding feature vector in the time series of the head state semantic coding feature vector to obtain a sequence of feature energy level coefficients; arranging the sequence of feature energy level coefficients into a feature energy level coefficient input vector; performing one-dimensional convolution encoding on the feature energy level coefficient input vector to obtain a feature energy level radiation association vector; performing point convolution encoding and based on The probabilistic mapping of the function is used to obtain a characteristic energy level radiation associated weight vector; the characteristic energy level radiation associated weight vector is used as the weight vector, and the time series of the head state semantic coding feature vector is weighted and residual processed to obtain the time series of the enhanced head state semantic coding feature vector.

[0026] Among them, the process of calculating the characteristic energy level coefficients of each head state semantic coding feature vector in the time series of the head state semantic coding feature vector to obtain a sequence of characteristic energy level coefficients includes: calculating the weighted sum between the variance of the head state semantic coding feature vector and the bias term to obtain a first intermediate term, and calculating the square of the difference between the maximum value and the mean of the head state semantic coding feature vector, the variance of the head state semantic coding feature vector and the bias term to obtain a second intermediate term; calculating the first intermediate term divided by the second intermediate term to obtain the characteristic energy level coefficient.

[0027] In summary, in the above embodiment, the time series of the head state semantic coding feature vector is input into the semantic association enhancer based on the feature energy level measurement associated radiation to obtain the time series of the enhanced head state semantic coding feature vector, including: processing the time series of the head state semantic coding feature vector with the following semantic association enhancement formula to obtain the time series of the enhanced head state semantic coding feature vector, wherein the semantic association enhancement formula is: in, represents the time series of the semantic encoding feature vector of the head state, The value of is the number of feature vectors in the time series of the head state semantic encoding feature vector, , , and respectively represent the first, second, and third and head state semantic encoding feature vector, For the said The mean of each eigenvalue of the semantic encoding feature vector of the head state, For the said The variance of each eigenvalue of the semantic encoding feature vector of the head state, Indicates the The maximum value of each eigenvalue of the semantic encoding feature vector of the head state, is the bias term, , , and respectively represent the first, second, and third and The characteristic energy level coefficient of the head state semantic encoding feature vector, represents the characteristic energy level coefficient input vector, represents a one-dimensional convolution operation, is the scale of the one-dimensional convolution kernel, represents the characteristic energy level radiation correlation vector, represents the point convolution operation, express function, represents the characteristic energy level radiation associated weight vector, represents dot product, A time series representing the semantically encoded feature vector of the enhanced head state.

[0028] In particular, the S4 determines whether the monitored student object is focused based on the head state temporal behavior pattern characteristics of the time series of the enhanced head state semantic encoding feature vector. In particular, in a specific example of the present application, Figure 5 As shown, the S4 includes: S41, performing a global temporal behavior pattern analysis on the time series of the enhanced head state semantic encoding feature vector to obtain a head state temporal behavior pattern semantic representation vector; S42, inputting the head state temporal behavior pattern semantic representation vector into a classifier-based state recognizer to obtain a recognition result, and the recognition result is used to indicate whether the monitored student object is focused.

[0029] Specifically, the S41 performs a global temporal behavior pattern analysis on the time series of the enhanced head state semantic encoding feature vector to obtain a head state temporal behavior pattern semantic representation vector. In the technical solution of the present application, the time series of the enhanced head state semantic encoding feature vector is input into a head state temporal behavior pattern feature extractor based on a Bi-LSTM model to obtain the head state temporal behavior pattern semantic representation vector. In order to capture the dynamic change pattern of the student's head state in the time dimension and fully understand the temporal context information of the student's head state, a head state temporal behavior pattern feature extractor based on a Bi-LSTM model is further used to perform global temporal modeling on the time series of the enhanced head state semantic encoding feature vector, and the bidirectional information transmission capability of the Bi-LSTM model is used to simultaneously capture the feature dependency of the previous and next time steps in the time series, and information transmission and fusion are performed on each enhanced head state semantic encoding feature vector in the time dimension, thereby obtaining a head state temporal behavior pattern semantic representation vector. It is worth mentioning that Bi-LSTM (Bidirectional Long Short-Term Memory) is a deep learning model used to process sequence data, which has the ability to remember long-term dependencies and bidirectional information transmission. Bi-LSTM combines forward LSTM and backward LSTM, and can simultaneously consider the past and future information of sequence data, thereby more comprehensively understanding the contextual relationship in the sequence.

[0030] Specifically, in S42, the semantic representation vector of the head state temporal behavior pattern is input into a state recognizer based on a classifier to obtain a recognition result, and the recognition result is used to indicate whether the monitored student object is focused. It should be understood that during the training process, the classifier can adopt a supervised learning method, use pre-labeled concentration label data for training and learning, and optimize the model parameters through back propagation, so that the classifier can accurately learn and establish a decision boundary related to the student's concentration to distinguish whether the student is focused on listening. In practical applications, when the semantic representation vector of the head state temporal behavior pattern is input into the state recognizer, the classifier can use the decision rules learned through training to classify and map the semantic representation vector of the head state temporal behavior pattern, and output a prediction result of whether the current student is focused. Thereby achieving real-time evaluation of the concentration of the monitored student object.

[0031] In the technical solution of the present application, each head state text description in the time series of the head state text description is a text semantic expression of the head state monitoring image of each head state monitoring key frame in the time series of the head state monitoring key frame based on the encoder-decoder structure, which can reduce the amount of data processing, but will also cause the time series of the head state text description to be offset in the semantic space relative to the time series of the head state monitoring key frame. Furthermore, after semantic encoding of each head state text description in the time series of the head state text description, semantic association enhancement based on feature energy level metric association radiation, and head state temporal behavior pattern feature extraction based on the Bi-LSTM model, the obtained semantic representation vector of the head state temporal behavior pattern will also lack the prior-posteriori class probability causal association relative to the semantic distribution representation of the source image due to the source domain modal transformation, resulting in the overflow of the recognition result obtained by the semantic representation vector of the head state temporal behavior pattern through the classifier-based state recognizer, affecting its accuracy. In particular, in a preferred embodiment, inputting the semantic representation vector of the head state temporal behavior pattern into a classifier-based state recognizer to obtain a recognition result specifically includes: Passing the head state temporal behavior pattern semantic representation vector through a probabilistic activation function to obtain a probabilistic head state temporal behavior pattern semantic representation vector; Determine the concentration probability value indicating whether the monitored student object is concentrating, obtained by the semantic representation vector of the head state temporal behavior pattern through a state identifier based on a classifier; After performing point subtraction on the unit feature vector and the probabilistic semantic representation vector of the head state temporal behavior pattern, the power function with the concentration probability value as the base is calculated, and further multiplied by the probabilistic semantic representation vector of the head state temporal behavior pattern to obtain the probabilistic semantic convergence limiting feature vector of the head state temporal behavior pattern; Taking the absolute value of each eigenvalue of the probabilistic semantic representation vector of the head state temporal behavior pattern as the base, calculating a power function with the inverse of the concentration probability value as the exponent, and summing all eigenvalues ​​of the probabilistic semantic representation vector of the head state temporal behavior pattern to obtain a semantic convergence bias eigenvalue of the head state temporal behavior pattern; The optimized head state temporal behavior pattern semantic representation vector is obtained by multiplying the probabilistic head state temporal behavior pattern semantic convergence limiting feature vector and the head state temporal behavior pattern semantic convergence bias feature value by a weight as a hyperparameter; and The optimized semantic representation vector of the head state temporal behavior pattern is input into the classifier-based state identifier to obtain the identification result.

[0032] Wherein, the semantic representation vector of the probabilistic head state temporal behavior pattern is recorded as , the focus probability value is , then the optimized semantic representation vector of the head state temporal behavior pattern is It is expressed as: in, is the semantic representation vector of the probabilistic head state temporal behavior pattern, represents the unit eigenvector with all eigenvalues ​​being one, is the weight as a hyperparameter, is the first eigenvalues, represents the probability value of focus, is a power function that calculates the feature value of each position in the feature vector with the focus probability value as an exponent, It means point multiplication by position. It means to make a difference by position. It means adding by position. It is the semantic representation vector of the optimized head state temporal behavior pattern.

[0033] That is, for the semantic representation vector of the head state temporal behavior pattern, based on the convergence limit of the Cauchy-Hadamard form power series probability distribution, the feature set of the semantic representation vector of the head state temporal behavior pattern is represented by a formal power series based on the convergence probability, and a set convergence bounded bias represented by the convergence radius based on each eigenvalue of the semantic representation vector of the head state temporal behavior pattern is added. Under a unified probability convergence limit standard, this can avoid the invalid overflow of the classification result caused by the deviation of the distribution pattern of the semantic representation vector of the head state temporal behavior pattern during the regression process of the classifier-based state recognizer due to the gradient propagation of the model parameters based on the convergence probability, thereby improving the accuracy of the recognition result obtained by the classifier-based state recognizer.

[0034] In particular, in S5, in response to the recognition result that the monitored student object is not focused, a reminder subtitle is generated to prompt the teacher to pay attention to the learning status of the monitored student object. That is, in response to the recognition result that the monitored student object is not focused, a reminder subtitle is generated to prompt the teacher to pay attention to the learning status of the monitored student object, so as to take corresponding teaching management intervention measures, such as reminding students to pay attention, adjusting teaching strategies, etc., to improve the quality and effect of online teaching.

[0035] It should be understood that before using the above-mentioned neural network model for inference, it is necessary to train the head semantic text expresser based on the encoder-decoder structure, the semantic association enhancer based on the feature energy level measurement associated radiation, the head state temporal behavior pattern feature extractor based on the Bi-LSTM model, and the state recognizer based on the classifier. That is to say, in the artificial intelligence-based online education management method of the present application, it also includes a training phase for training the head semantic text expresser based on the encoder-decoder structure, the semantic association enhancer based on the feature energy level measurement associated radiation, the head state temporal behavior pattern feature extractor based on the Bi-LSTM model, and the state recognizer based on the classifier.

[0036] Figure 3 FIG. 1 is a flow chart of the training phase of the online education management method based on artificial intelligence according to an embodiment of the present application. Figure 3 As shown, according to the embodiment of the present application, the online education management method based on artificial intelligence includes: a training stage, including: S110, obtaining training data, wherein the training data includes a training head state monitoring video of a monitored student object collected by a camera, and a real value of whether the monitored student object is focused; S120, performing key frame sampling on the training head state monitoring video at a predetermined sampling frequency to obtain a time series of training head state monitoring key frames; S130, inputting each training head state monitoring key frame in the time series of the training head state monitoring key frames into the head semantic text expressor based on the encoder-decoder structure to obtain a time series of training head state text descriptions; S140, semantically encoding each training head state text description in the time series of the training head state text descriptions to obtain a time series of training head state semantic encoding feature vectors; S150, The time series of the training head state semantic encoding feature vector is input into the semantic association enhancer based on feature energy level measurement associated radiation to obtain the time series of the training enhanced head state semantic encoding feature vector; S160, the time series of the training enhanced head state semantic encoding feature vector is input into the head state temporal behavior pattern feature extractor based on the Bi-LSTM model to obtain the training head state temporal behavior pattern semantic representation vector; S170, the training head state temporal behavior pattern semantic representation vector is input into the classifier-based state recognizer to obtain the classification loss function value; S180, the classification loss function value is used to train the head semantic text expresser based on the encoder-decoder structure, the semantic association enhancer based on feature energy level measurement associated radiation, the head state temporal behavior pattern feature extractor based on the Bi-LSTM model and the state recognizer based on the classifier.

[0037] In summary, the online education management method based on artificial intelligence according to the embodiment of the present application is explained, which uses computer vision technology based on deep learning to perform semantic analysis on the student head state monitoring video collected by the camera, and mines the semantic feature representation of the head state of each key frame in the video, and then analyzes the global temporal behavior pattern of the student's head state, so as to intelligently evaluate whether the student is concentrating on listening, so as to implement effective teaching management intervention. In this way, real-time monitoring of student concentration in the online education environment can be achieved, the teacher's perception of the student's learning status can be improved, and the student's distraction behavior can be discovered and corrected in time, thereby improving the quality and effect of online teaching.

[0038] Furthermore, an online education management system based on artificial intelligence is also provided.

[0039] Figure 6 FIG. 1 is a block diagram of an online education management system based on artificial intelligence according to an embodiment of the present application. Figure 6 As shown, according to the embodiment of the present application, the online education management system 300 based on artificial intelligence includes: a monitoring video acquisition module 310, which is used to obtain the head state monitoring video of the monitored student object collected by the camera; a key frame semantic coding module 320, which is used to perform key frame semantic coding on the head state monitoring video to obtain a time series of head state semantic coding feature vectors; a semantic association enhancement module 330, which is used to input the time series of the head state semantic coding feature vector into a semantic association enhancer based on feature energy level measurement associated radiation to obtain a time series of enhanced head state semantic coding feature vectors; a concentration state determination module 340, which is used to determine whether the monitored student object is focused based on the head state temporal behavior pattern characteristics of the time series of the enhanced head state semantic coding feature vectors; a reminder subtitle generation module 350, which is used to generate reminder subtitles in response to the recognition result that the monitored student object is not focused to prompt the teacher to pay attention to the learning status of the monitored student object.

[0040] As described above, the online education management system 300 based on artificial intelligence according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an online education management algorithm based on artificial intelligence. In one possible implementation, the online education management system 300 based on artificial intelligence according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the online education management system 300 based on artificial intelligence can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the online education management system 300 based on artificial intelligence can also be one of the many hardware modules of the wireless terminal.

[0041] Alternatively, in another example, the artificial intelligence-based online education management system 300 and the wireless terminal may also be separate devices, and the artificial intelligence-based online education management system 300 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0042] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An online education management method based on artificial intelligence, characterized in that: include: Obtaining a head status monitoring video of a monitored student object collected by a camera; Performing key frame semantic coding on the head state monitoring video to obtain a time series of head state semantic coding feature vectors; Inputting the time series of the head state semantic encoding feature vectors into a semantic association enhancer based on feature energy level measurement association radiation to obtain a time series of enhanced head state semantic encoding feature vectors; Determining whether the monitored student object is focused based on the head state temporal behavior pattern characteristics of the time series of the enhanced head state semantic encoding feature vector; In response to the identification result that the monitored student object is not concentrating, a reminder subtitle is generated to prompt the teacher to pay attention to the learning status of the monitored student object.

2. The online education management method based on artificial intelligence according to claim 1 is characterized in that: Performing key frame semantic coding on the head state monitoring video to obtain a time series of head state semantic coding feature vectors, including: Performing key frame sampling on the head state monitoring video at a predetermined sampling frequency to obtain a time sequence of head state monitoring key frames; Each head state monitoring key frame in the time sequence of the head state monitoring key frames is respectively expressed in a semantic text form to obtain a time sequence of head state text description; Semantic coding is performed on each head state text description in the time series of the head state text description to obtain a time series of the head state semantic coding feature vector.

3. The online education management method based on artificial intelligence according to claim 2 is characterized in that: The time sequence of the head state monitoring key frames in the time sequence of the head state monitoring key frames is semantically expressed in text form to obtain a time sequence of the head state text description, including: Each head state monitoring key frame in the time sequence of the head state monitoring key frames is input into a head semantic text expressor based on an encoder-decoder structure to obtain the time sequence of the head state text description.

4. The online education management method based on artificial intelligence according to claim 3 is characterized in that: Inputting the time series of the head state semantic encoding feature vector into a semantic association enhancer based on feature energy level measurement association radiation to obtain an enhanced time series of the head state semantic encoding feature vector, including: Calculating the characteristic energy level coefficient of each head state semantic encoding feature vector in the time series of the head state semantic encoding feature vector to obtain a sequence of characteristic energy level coefficients; Arranging the sequence of characteristic energy level coefficients into a characteristic energy level coefficient input vector; Performing one-dimensional convolution coding on the characteristic energy level coefficient input vector to obtain a characteristic energy level radiation correlation vector; The characteristic energy level radiation correlation vector is point convolution encoded and based on Probabilistic mapping of functions to obtain characteristic energy level radiation associated weight vectors; The characteristic energy level radiation associated weight vector is used as a weight vector, and weighted processing and residual processing are performed on the time series of the head state semantic coding feature vector to obtain the time series of the enhanced head state semantic coding feature vector.

5. The online education management method based on artificial intelligence according to claim 4 is characterized in that: Calculating the characteristic energy level coefficient of each head state semantic encoding feature vector in the time series of the head state semantic encoding feature vector to obtain a sequence of characteristic energy level coefficients, including: Calculate the weighted sum of the variance and the bias term of the head state semantic coding feature vector to obtain a first intermediate term, and calculate the weighted sum of the square of the difference between the maximum value and the mean of the head state semantic coding feature vector, the variance of the head state semantic coding feature vector, and the bias term to obtain a second intermediate term; The first intermediate term divided by the second intermediate term is calculated to obtain the characteristic energy level coefficient.

6. The online education management method based on artificial intelligence according to claim 5 is characterized in that: Determining whether the monitored student object is concentrating based on the head state temporal behavior pattern characteristics of the time series of the enhanced head state semantic encoding feature vector includes: Performing a global temporal behavior pattern analysis on the time series of the enhanced head state semantic encoding feature vector to obtain a head state temporal behavior pattern semantic representation vector; The semantic representation vector of the head state temporal behavior pattern is input into a classifier-based state identifier to obtain a recognition result, and the recognition result is used to indicate whether the monitored student object is concentrating.

7. The online education management method based on artificial intelligence according to claim 6 is characterized in that: Performing a global temporal behavior pattern analysis on the time series of the enhanced head state semantic encoding feature vector to obtain a head state temporal behavior pattern semantic representation vector, including: The time series of the enhanced head state semantic encoding feature vector is input into the head state temporal behavior pattern feature extractor based on the Bi-LSTM model to obtain the head state temporal behavior pattern semantic representation vector.

8. The online education management method based on artificial intelligence according to claim 7 is characterized in that: It also includes a training step: training the head semantic text expresser based on the encoder-decoder structure, the semantic association enhancer based on feature energy level metric association radiation, the head state temporal behavior pattern feature extractor based on the Bi-LSTM model, and the state recognizer based on the classifier.

9. The online education management method based on artificial intelligence according to claim 8 is characterized in that: The training step comprises: Acquire training data, wherein the training data includes a training head state monitoring video of a monitored student object collected by a camera, and a real value of whether the monitored student object is focused; Sampling the training head state monitoring video for key frames at a predetermined sampling frequency to obtain a time sequence of training head state monitoring key frames; Inputting each training head state monitoring key frame in the time sequence of the training head state monitoring key frame into the head semantic text expressor based on the encoder-decoder structure to obtain a time sequence of training head state text description; semantically encoding each training head state text description in the time series of the training head state text description to obtain a time series of training head state semantic encoding feature vectors; Inputting the time series of the training head state semantic encoding feature vectors into the semantic association enhancer based on feature energy level measurement association radiation to obtain the time series of the training enhanced head state semantic encoding feature vectors; Inputting the time series of the training enhanced head state semantic encoding feature vector into the head state temporal behavior pattern feature extractor based on the Bi-LSTM model to obtain a training head state temporal behavior pattern semantic representation vector; Inputting the semantic representation vector of the training head state temporal behavior pattern into the classifier-based state recognizer to obtain a classification loss function value; The classification loss function value is used to train the head semantic text expresser based on the encoder-decoder structure, the semantic association enhancer based on feature energy level metric association radiation, the head state temporal behavior pattern feature extractor based on the Bi-LSTM model, and the classifier-based state recognizer.

10. An online education management system based on artificial intelligence, characterized in that: include: A monitoring video acquisition module is used to acquire the head status monitoring video of the monitored student object collected by the camera; A key frame semantic coding module, used for performing key frame semantic coding on the head state monitoring video to obtain a time series of head state semantic coding feature vectors; A semantic association enhancement module, used for inputting the time series of the head state semantic encoding feature vector into a semantic association enhancer based on feature energy level measurement association radiation to obtain a time series of enhanced head state semantic encoding feature vector; A concentration state determination module, used to determine whether the monitored student object is concentrating based on the head state temporal behavior pattern characteristics of the time series of the enhanced head state semantic encoding feature vector; The reminder subtitle generation module is used to generate reminder subtitles to prompt the teacher to pay attention to the learning status of the monitored student object in response to the recognition result that the monitored student object is not focused.

Images