A teaching behavior analysis system and method based on computing power network

Abstract

The present application relates to a kind of teaching behavior analysis system and method based on computing power network, including: algorithm network cloud platform layer, communication network layer, edge layer and the Web end for user interaction.Algorithm network cloud platform layer is used for: data annotation;The training of target detection model;Storage dataset, image, target detection model file and data flow in each storage node conversion.Communication network layer is used for: algorithm network cloud platform layer and edge layer carry out data, information interaction;Edge layer is used for: inference to real-time video stream;The Web end for user interaction is used for: provide the service of uploading data, selecting model training mode, viewing model training result and teaching behavior analysis report for user.The present application carries out long-term evaluation to a course, gives a comprehensive course concentration analysis report at the end of semester, is of great help to teacher adjustment syllabus, also makes school to the teaching evaluation of teacher more objective.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of AI model training and smart education technologies based on computing power networks. Specifically, it relates to a teaching behavior analysis system and method based on computing power networks. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence (AI) technology, the informatization of education has undergone tremendous changes. The latest achievements of AI are gradually being applied to various aspects of education, such as intelligent teaching, digital assessment, personalized learning, and data management. Against this backdrop, in-depth analysis of students' classroom teaching behavior has become possible. Traditional analysis methods, such as classroom observation and questionnaires, generally suffer from certain limitations: low efficiency, high error rates, time-consuming and labor-intensive methods, over-reliance on experts, low analytical efficiency, and difficulty in scaling. They cannot comprehensively reflect the teaching status of teachers and students during the teaching process, nor are they convenient for real-time data analysis and utilization. However, combining AI with classroom behavior analysis can quickly and accurately generate teaching behavior analysis reports, helping teachers dynamically adjust their teaching plans, significantly reducing the time and effort teachers spend on teaching analysis, and providing data support for personalized learning recommendations for students and recording the growth trajectory of instructors' teaching levels, thus achieving precision teaching.

[0003] With the introduction of the "Education Informatization 2.0 Action Plan," many universities have focused on intelligent teaching behavior analysis. However, several challenges remain: First, developing a teaching behavior analysis system requires significant human, material, and time resources. Training the analysis model demands substantial GPU computing power, resources generally unavailable to most universities. Even those universities with sufficient GPU power cannot develop such systems due to cost considerations, hindering the provision of real-time and accurate analysis reports and impacting teaching progress, thus preventing the achievement of precision teaching requirements. Second, some universities, even those aiming to develop such systems, lack sufficient GPU resources. Model training requires substantial computing power, which slows down model updates and results in inaccurate analysis reports, potentially misleading teachers. Third, different universities have varying time requirements for generating teaching behavior analysis reports and different GPU computing power requirements for training models. Some prioritize high-speed (low-latency) computing power, while others prioritize cost-effectiveness. Relying solely on cloud computing or simple cloud-edge collaborative architectures cannot flexibly meet these diverse needs. Fourth, some existing teaching behavior analysis systems use a basic cloud-edge technology architecture, which has problems such as inflexible scheduling of computing resources and insufficient performance. Therefore, it is necessary to adopt a teaching behavior analysis method based on computing power networks.

[0004] Computing power networks are a new network architecture oriented towards the convergence of computing and networks. Through the coordinated scheduling of computing and network resource status, different application services are scheduled to the optimal computing nodes via the optimal path, ensuring the optimal utilization of network and computing resources. For teaching behavior analysis systems, computing power networks can more flexibly schedule computing and storage resources according to user needs, so as to better meet user requirements and ensure full utilization of resources.

[0005] Currently, computing networks have not yet been maturely applied in the field of teaching behavior analysis. On the one hand, as mentioned above, computing networks have almost unlimited computing and storage resources, which are sufficient to support the scale of users in the future. On the other hand, even if different users have different training scales and different needs for timeliness and price, the computing network can ensure that the utilization rate of network and storage resources is optimal through its own scheduling, so as to achieve the best user experience while avoiding waste of resources. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to propose a teaching behavior analysis method and system based on a computing network. The computing network center, based on the GPU computing power of university users and their own choices, schedules the optimal GPU and storage resources. Utilizing a unified storage and data transfer platform, data is transferred to the corresponding computing nodes during network downtime. The optimal computing resources scheduled by the computing network are used to train the target detection model. The trained model is then used to infer the classroom videos that the user needs to analyze, generating a teaching behavior analysis report. Finally, the inferred videos and analysis report are presented to the user, enabling them to make teaching adjustments based on the corresponding reports.

[0007] Terminology Explanation:

[0008] 1. Computing power network is a new type of information infrastructure that allocates and flexibly schedules computing, storage, and network resources on demand among the cloud, network, and edge according to business needs. It deeply integrates edge computing nodes, cloud computing nodes, and various network resources, including wide area networks, reducing the management complexity of edge computing nodes. Through centralized control or distributed scheduling methods, it coordinates with the computing and storage resources of cloud computing nodes and the network resources of wide area networks to form a new generation of information infrastructure. It provides customers with overall computing power services including computing, storage, and connectivity, and provides flexible and schedulable on-demand services according to business characteristics.

[0009] 2. Object Storage Service (OSS), also known as object-based storage, provides data storage services in the form of objects on a distributed system. It offers RESTful API data read / write interfaces and rich SDK interfaces, and is often provided as a network service. Object storage is suitable for storing unstructured data and features multi-node, flat structure, and elastic scaling. This means that regardless of the user's geographical location, data can be stored in the nearest data center, accelerating data access. All data is stored at the same level; users do not need to know where the data is located, but can quickly retrieve the data using a Uniform Resource Identifier (URI).

[0010] 3. Kubernetes. Kubernetes is an open-source system for automatically deploying, scaling, and managing containerized applications. In Kubernetes, multiple containers can be created, each running an application instance. Then, through built-in load balancing strategies, the management, discovery, and access of this group of application instances can be achieved.

[0011] 4. KubeEdge: KubeEdge is an open-source system that extends natively containerized business processes and device management to hosts on Edge. Built on Kubernetes, it provides core infrastructure support for networking, application deployment, and metadata synchronization between the cloud and the edge.

[0012] 5. The YOLOv5 algorithm is a single-stage object detection algorithm that employs an end-to-end approach using a regression strategy. It applies a single convolutional neural network (CNN) to the entire image, dividing the image into a grid and predicting the class probability and bounding box for each grid. It balances detection speed and accuracy, enabling very accurate real-time object detection.

[0013] 6. An API (Application Programming Interface) is a set of predefined functions designed to provide applications and developers with the ability to access a set of routines based on certain software or hardware, without needing to access the source code or understand the details of the internal workings.

[0014] 7. OpenCV is an open-source computer vision library that provides tools and algorithms for processing images and videos.

[0015] 8. HDFS storage is a distributed file system designed to run on commodity hardware; HDFS is a highly fault-tolerant system that provides high-throughput data access and is well-suited for applications on large-scale datasets.

[0016] 9. NFS storage: NFS is a network file system protocol based on TCP / IP. By using the NFS protocol, clients can access shared resources on remote servers as if they were local directories. For most load-balanced clusters, using the NFS protocol to share data storage is a common method.

[0017] 10. Parallel file storage is a technology that can provide high-performance file system services, meeting the file system requirements of high-performance applications.

[0018] 11. WebSocket protocol: WebSocket is a transmission protocol proposed to solve the problem of bidirectional communication between the server and the client. It enables the client and the server to push and receive messages to each other. It is a full-duplex communication network technology. Either party can establish a connection to push data to the other party. WebSocket only needs to establish a connection once and can maintain it indefinitely.

[0019] The technical solution of this invention is as follows:

[0020] A teaching behavior analysis system based on computing power network includes: a computing network cloud platform layer, a communication network layer, an edge layer, and a web terminal for user interaction.

[0021] The cloud platform layer is used for: data annotation; training of object detection models; storing datasets, images, object detection model files, and data flow between various storage nodes.

[0022] The communication network layer is used for data and information exchange between the cloud platform layer and the edge layer.

[0023] Edge layer, used for: inference of real-time video streams;

[0024] The web-based interface for user interaction is used to provide users with services such as uploading data, selecting model training methods, viewing model training results, and receiving teaching behavior analysis reports.

[0025] According to a preferred embodiment of the present invention, the computing cloud platform layer includes a cloud core module of kubeedge, a cloud communication module, computing resources, a unified data storage and data transfer platform, and an automatic data labeling module;

[0026] The cloud core module is a component that interacts with the cloud. It is used to send instructions from the cloud to the edge layer and is also responsible for receiving events reported from the edge layer to the cloud.

[0027] The cloud communication module acts as a network proxy for the cloud, through which all cloud traffic passes; enabling communication between the cloud core module and the edge layer.

[0028] The computing resources of the computing network are used for training the target detection model;

[0029] The unified data storage and data transfer platform is used to store, in object storage mode, the original video sources uploaded by university users, the data used for training the target detection model after being labeled by the automatic data labeling module, the historical and best versions of the target detection model, the inference results of the target detection model in the edge layer, and the teaching behavior analysis reports generated for users; and to transfer the data to the best storage node according to the user's needs.

[0030] The automatic data annotation module is used to: segment the video information provided by the user into images, automatically annotate the behavior of students in the images, and export a training set that meets the training requirements of the object detection model.

[0031] More preferably, the unified data storage and data transfer platform is a storage platform in the computing network, including object storage, HDFS storage, NFS storage, and parallel file storage. The unified data storage and data transfer platform controls all storage nodes and enables data to flow between different computing nodes, and provides API interfaces for data access.

[0032] According to a preferred embodiment of the present invention, the edge layer includes the kubeedge edge core module, the edge communication module, and computing resources for university users;

[0033] The edge core module is used to manage the components of the edge load. It is responsible for receiving and executing instructions sent from the cloud, that is, managing the entire life cycle of the edge load, and reporting the status of the edge load to the cloud in the form of events.

[0034] The edge communication module acts as a network proxy for the edge layer, through which all traffic entering and leaving the edge layer passes; enabling communication between the cloud core module and the edge layer.

[0035] The computing resources of the university users, namely their GPU devices, are used to execute inference tasks distributed from the cloud and store the inference results in the unified data storage and data transfer platform.

[0036] According to a preferred embodiment of the present invention, the cloud communication module and the edge communication module communicate bidirectionally via the WebSocket protocol.

[0037] A teaching behavior analysis method based on computing power networks, implemented through the aforementioned teaching behavior analysis system based on computing power networks, includes:

[0038] Step 1: Edge layer device deployment: Deploy the corresponding edge core modules and edge communication modules according to the training model method selected by the user;

[0039] Step 2: Source data storage and processing: Store the original teaching video data uploaded by users into the unified storage and data transfer platform of the computing power network, and process the original teaching video data in the unified storage and data transfer platform;

[0040] Step 3: Object detection model training method: The computing power network allocates suitable cloud servers or edge servers based on the user selection in Step 1, the size of the training task, and the geographical location of the user. The training data stored in the unified storage and data transfer platform in Step 2 is transferred to the most suitable server allocated by the computing power network in Step 3 to train the object detection model. There are three training methods depending on the allocated server. The object detection model is trained after the training method is determined.

[0041] Step 4: Use the trained object detection model to infer the real-time classroom video stream and generate a classroom behavior analysis report.

[0042] According to a preferred embodiment of the present invention, the deployment method of step 1 includes the following two methods:

[0043] The first option is: if the user selects that the training and inference processes of the object detection model are both completed locally at the edge layer, or the training process of the object detection model is completed at the computing network while the inference process is completed locally, then an edge communication module is deployed at the edge layer to communicate with the cloud and obtain the object detection model training and inference tasks.

[0044] The second option is: if the user selects that the training and inference tasks of the object detection model are both completed on the computing network, then there is no need to deploy an edge communication module at the edge layer, and the computing network directly pushes the inference results and analysis reports to the front-end page.

[0045] According to a preferred embodiment of the present invention, source data storage and processing includes:

[0046] Step 2.1: Source data storage: Call the API interface provided by the unified storage and data transfer platform of the computing power network to store the original video source uploaded by the user and the course hours, classrooms, teachers and teaching plans in the teaching system in the unified storage and data transfer platform in the form of object storage;

[0047] Step 2.2: Data Processing: Call the API interface of the automatic data labeling module to read data from the unified storage and data flow platform, import the data, automatically extract the imported original video source, and automatically label the extracted data according to the user's needs by calling the API interface provided by the automatic data labeling module, labeling the student's actions in each extracted image with corresponding labels, and dividing the labeled data into training set and test set.

[0048] Step 2.3: Call the API interface of the automatic data labeling module to export the labeled data and transfer it to the unified data storage and data transfer platform of the computing power network in the form of object storage.

[0049] According to a preferred embodiment of the present invention, the three methods for training the object detection model include the following:

[0050] The first approach is: if the server matched by the computing power network for the training and inference process of the object detection model is a cloud server, then the training data is directly obtained from the unified storage and data transfer platform of the computing power network, and the object detection model is trained.

[0051] The second scenario is: if the server matched by the computing power network for the training and inference process of the object detection model is an edge server, then the edge server first obtains the training data, inference data and the original object detection model image from the unified storage and data transfer platform and stores them in the local file storage system. The cloud sends training tasks to the edge server through the kubeedge management platform, and the edge server trains the object detection model.

[0052] The third scenario is as follows: If the server used for training the object detection model in the computing power network is a cloud server, while the server used for model inference is an edge server, then the cloud server obtains the training data and the original object detection model image from the unified storage and data transfer platform of the computing power network, trains the object detection model with the training data, and finally generates an image of the object detection model with the best training result and sends it to the unified storage and data transfer platform.

[0053] A preferred embodiment of the object detection model training method according to the present invention includes:

[0054] The target detection model uses YOLOv5, which includes an input, a backbone network, a neck network, and an output head.

[0055] Data augmentation is performed at the input end using the Mosaic method;

[0056] The backbone network performs slicing operations, inputting a 608×608×3 three-channel image into the Focus structure. After slicing, it is first transformed into a 304×304×12 feature map. Then, after convolution operations using 32 convolution kernels, a final 304×304×32 feature map is obtained.

[0057] In the FPN layer of the Neck network, the feature information of the high-level layer and the feature information of the low-level layer are fused through the upsampling operation to calculate the predicted feature map. After the FPN layer, a feature pyramid is added from bottom to top, which includes two PAN structures. Through the downsampling operation, the feature information of the low-level layer and the feature information of the high-level layer are fused to output the predicted feature map.

[0058] After optimizing the output Head to ensure that the predicted bounding boxes and the ground truth bounding boxes do not intersect, the prediction results are output.

[0059] According to a preferred embodiment of the present invention, the three training methods also correspond to three inference methods:

[0060] The first method is: when the target detection model is trained in the first way, the trained target detection model is used to infer the real-time classroom video stream, and the inferred video stream and classroom behavior analysis report are stored in a unified storage and data transfer platform.

[0061] The second method is as follows: When the target detection model is trained in the second way, the cloud sends inference tasks to the edge through the kubeedge management platform. The edge performs inference on the real-time classroom video stream and stores the inference video stream and classroom behavior analysis report in the unified storage and data transfer platform of the computing power network.

[0062] The third method is as follows: When the object detection model is trained in the third method, the edge server obtains the real-time classroom video stream and the trained object detection model image from the unified storage and data transfer platform. The cloud server then issues inference tasks through the kubeedge management platform, and the edge server sends the inference video stream and analysis report to the unified storage and data transfer platform.

[0063] According to a preferred embodiment of the present invention, in step 4, the unified storage and data transfer platform pushes the obtained reasoning video stream and classroom behavior analysis report to the front end. A visual analysis report is generated for each class's real-time video stream. Furthermore, a long-term evaluation is conducted for a course, combining the reasoning results of each class with the timetable information. As the course progresses, a total focus report for the course is given at the end of the semester.

[0064] The beneficial effects of this invention are as follows:

[0065] 1. This invention discloses a teaching behavior analysis method based on computing power networks, allowing users to select the cloud-edge collaborative training method and the student behaviors to be analyzed according to the configuration of their edge devices. The training method is divided into three types based on GPU computing power resources:

[0066] The first type is where edge devices do not have GPU computing power. The training and inference of deep learning models are carried out on cloud servers. The edge device only needs to upload the monitoring video for training and the teaching video stream that needs to generate the analysis report to obtain the teaching behavior analysis report.

[0067] The second type is where the edge device has GPU computing power, but it is not enough to complete the entire training and inference process. The edge device needs to upload the monitoring video for training to the system. The cloud will label the monitoring video data and train the teaching behavior analysis model. Then the trained model will be sent to the edge device. After the edge device obtains the model, it will use the model to infer the real-time classroom video stream and return the inferred video stream to the cloud. The cloud will then give an analysis report based on the inferred video stream.

[0068] The third scenario involves edge computing power sufficient for model training and inference. In this case, the cloud will send the original model image to the edge, where the model will be trained and inferred. Finally, the inference video stream will be returned to the cloud, which will then provide an analysis report.

[0069] The above three methods cover all user situations, and the scheduling of resources such as GPU computing power and big data storage based on the computing power network enables the present invention to meet the personalized needs of different university users for teaching behavior analysis services.

[0070] 2. This invention establishes a web-based teaching behavior analysis service and encapsulates the kubeedge backend interface to connect with the web-based service, providing users with a simple and easy-to-use visual interface. Users only need to upload the original video source on the page, select the behavior to be analyzed, the time for which the teaching behavior analysis results are needed, and the required GPU service, and they can obtain the teaching behavior analysis video and teaching behavior analysis report according to the selected time. The operation is simple and convenient, and has a good user experience.

[0071] 3. The video results and analysis reports generated by the teaching behavior analysis system of this invention can not only be used by schools to evaluate the teaching level of instructors, but also allow instructors to compare the objective data in the analysis reports with the teaching analysis videos in their spare time to improve their teaching level, and to adjust subsequent teaching plans in real time and accurately. This can continuously improve the quality of teachers' teaching, make students more interested in the courses, and increase the students' final exam pass rate. Therefore, teaching analysis can have a greater impact on both instructors and students.

[0072] 4. This invention is based on kubeedge, which is built on Kubernetes. It retains the Kubernetes management plane, redevelops the node agent, and simplifies the Kubernetes kubelet module. It significantly optimizes and reduces the resource consumption of edge components, achieving lightweight design. Even if the edge devices are relatively outdated, it will not affect the use of the system.

[0073] 5. Compared with traditional classroom teaching behavior analysis, this invention can conduct long-term evaluation of a course, combining the reasoning results of each lesson with the course's lesson plan, and providing a comprehensive course focus analysis report at the end of the semester. This greatly helps teachers adjust the syllabus and also makes the school's evaluation of teachers' teaching more objective. Attached Figure Description

[0074] Figure 1 This is a schematic diagram of the architecture of the teaching behavior analysis system based on computing power network of the present invention;

[0075] Figure 2 This is a flowchart illustrating the teaching behavior analysis method based on computing power networks of the present invention.

[0076] Figure 3 This is a schematic diagram of the neural network architecture of the YOLOv5 algorithm.

[0077] Figure 4 This is a schematic diagram of the web page where the user selects to use the computing network resource service in this invention;

[0078] Figure 5 This is a schematic diagram of the web page where the user selects to use local resource services in this invention;

[0079] Figure 6 This is a schematic diagram of the web page for the visual analysis report presented to the user in this invention. Detailed Implementation

[0080] Obviously, the examples listed in the specific embodiments are only a part of the examples of this invention, and not all of them. All other examples obtained by those skilled in the art based on the examples of this invention without inventive effort should fall within the protection scope of this invention. The invention is further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0081] Example 1

[0082] The first aspect of this invention provides a teaching behavior analysis system based on a computing network, such as... Figure 1 As shown, it includes: a computing cloud platform layer, a communication network layer, an edge layer, and a web interface for user interaction.

[0083] The cloud platform layer is used for: data annotation; training of object detection models; storing datasets, images, object detection model files, and data flow between various storage nodes.

[0084] The communication network layer adds SD-WAN leased lines to the existing network to solve the problem of acquiring classroom surveillance video; it is used for data and information exchange between the computing cloud platform layer and the edge layer.

[0085] The edge layer, deployed in the campus computer rooms of university users, is used for: inference of real-time video streams from classrooms;

[0086] The web-based interface for user interaction is used to provide users with services such as uploading data, selecting model training methods, viewing model training results, and receiving teaching behavior analysis reports.

[0087] Example 2

[0088] The teaching behavior analysis system based on computing power network described in Example 1 differs in that:

[0089] The cloud computing platform layer includes the cloud core module of kubeedge, the cloud communication module, computing resources, a unified data storage and data flow platform, and an automatic data labeling module;

[0090] The cloud core module is a component that interacts with the cloud, used to send instructions from the cloud to the edge layer, and also responsible for receiving events reported from the edge layer to the cloud. The cloud communication module acts as a network proxy for the cloud, through which all cloud traffic passes; it enables communication between the cloud core module and the edge layer. The computing network resources are used for training the object detection model. These resources, such as GPU servers, are optimally matched by the computing network based on the user's personalized task requirements. The unified data storage and data flow platform stores, in object storage mode, the original video sources uploaded by university users, data labeled by the automatic data labeling module for training the object detection model, historical and best versions of the object detection model, inference results from the edge layer object detection model, and teaching behavior analysis reports generated for users. It also allows data to flow to the optimal storage node according to user needs. The automatic data labeling module is used to segment user-provided video information into images, automatically label student behaviors in the images, and export a training set that meets the requirements for training the object detection model.

[0091] The unified data storage and data transfer platform is a storage platform in the computing network, including object storage (OSS), HDFS storage, NFS storage, and parallel file storage. The unified data storage and data transfer platform controls all storage nodes and enables data to flow between different computing nodes, and provides API interfaces for data access.

[0092] Example 3

[0093] The teaching behavior analysis system based on computing power network described in Example 2 differs in that:

[0094] The edge layer includes the kubeedge edge core module, the edge communication module, and computing resources for university users. The edge core module manages the edge workload, receiving and executing instructions from the cloud, managing the entire lifecycle of the edge workload, and reporting the status of the edge workload to the cloud in the form of events. The edge communication module acts as the network proxy for the edge layer, handling all traffic entering and leaving the edge layer, and enabling communication between the cloud core module and the edge layer. The computing resources for university users, namely their GPU devices, are used to execute inference tasks from the cloud and store the inference results in the unified data storage and data transfer platform.

[0095] The cloud communication module and the edge communication module communicate bidirectionally via the WebSocket protocol. The cloud acts as the WebSocket server, and the edge device acts as the WebSocket client. Commands to create pods from the cloud to the edge are sent via WebSocket. After the pods are deployed on the edge, the edge device also needs to report the pod's running status and log information to the cloud in real time. The web interface primarily provides users with functions such as uploading data, selecting model training methods, viewing model training results, and accessing teaching behavior analysis reports.

[0096] Example 4

[0097] A second aspect of this invention provides a method for analyzing teaching behavior based on computing power networks, such as... Figure 2 As shown, the teaching behavior analysis system based on computing power network described in any of Examples 1-3 is implemented. In this example, the GPU server selected by the user for the model training process is a cloud GPU server, while the GPU server for the model inference process is local at the edge. This includes:

[0098] Step 1: Edge Layer Device Deployment: Deploy the corresponding edge core module and edge communication module according to the training model selected by the user; The edge layer device deployment process includes: First, deploy the Golang environment, then use the wget command to download the keadm tool, and use the keadm join command based on the token in the cloud to deploy the edge layer device to the edge core module and edge communication module of kubeedge and add it to the kubeedge management platform.

[0099] Step 2: Source Data Storage and Processing: The raw teaching video data uploaded by users is stored in the unified storage and data transfer platform of the computing power network. The raw teaching video data is processed within this platform. The API interface of the automatic data annotation module is called to annotate the data in the unified storage platform according to user needs. The annotated dataset is then divided into training, testing, and validation sets before being stored in the unified storage and data transfer platform of the computing power network. This includes:

[0100] Step 2.1: Source Data Storage: Utilize the API provided by the unified storage and data transfer platform of the computing power network to store the user-uploaded original video sources, along with course hours, classrooms, teachers, and teaching plans from the teaching system, in object storage mode on the unified storage and data transfer platform. First, import the Object Storage Service (OSS) dependency, add the necessary parameters for accessing OSS to the application.yml configuration file, and write an OSS utility class (upload method) for reuse, reducing code volume. Second, store the original videos uploaded by web frontend users in object storage (OSS) mode on the computing power network's unified storage and data transfer platform.

[0101] Step 2.2: Data Processing: Call the API interface of the automatic data labeling module to read data from the unified storage and data flow platform, import the data, and write a Python program based on OpenCV as an automatic video screenshot tool to achieve an average of about 146 frames per second. Use the Python program based on OpenCV to automatically screenshot the original video in Step 2.1, and according to the user's needs, call the API interface provided by the automatic data labeling module to automatically label the captured data, and label the student's actions in each captured image. Divide the labeled data into training set and test set.

[0102] Step 2.3: Call the API interface of the automatic data labeling module to export the labeled data and transfer it to the unified data storage and data transfer platform of the computing power network in the form of object storage.

[0103] Step 3: Object detection model training method: The computing power network allocates suitable cloud servers or edge servers based on the user selection in Step 1, the size of the training task, and the geographical location of the user. The training data stored in the unified storage and data transfer platform in Step 2 is transferred to the most suitable server allocated by the computing power network in Step 3 to train the object detection model. There are three training methods depending on the allocated server. The object detection model is trained after the training method is determined.

[0104] The three methods for training an object detection model are as follows:

[0105] The first approach is: if the server matched by the computing power network for the training and inference process of the object detection model is a cloud server, then the training data is directly obtained from the unified storage and data transfer platform of the computing power network, and the object detection model is trained.

[0106] The second scenario is: if the server matched by the computing power network for the training and inference process of the object detection model is an edge server, then the edge server first obtains the training data, inference data and the original object detection model image from the unified storage and data transfer platform and stores them in the local file storage system. The cloud sends training tasks to the edge server through the kubeedge management platform, and the edge server trains the object detection model.

[0107] The third scenario is as follows: If the server used for training the object detection model in the computing power network is a cloud server, while the server used for model inference is an edge server, then the cloud server obtains the training data and the original object detection model image from the unified storage and data transfer platform of the computing power network, trains the object detection model with the training data, and finally generates an image of the object detection model with the best training result and sends it to the unified storage and data transfer platform.

[0108] Methods for training object detection models include:

[0109] To obtain the dataset: Modify the `data` parameter of the `parser.add_argument` function to the path where the training set is located.

[0110] The object detection model uses YOLOv5, and the network structure of YOLOv5 is as follows: Figure 3 As shown, YOLOv5 includes an input terminal, a backbone network, a neck network, and an output terminal (Head).

[0111] Mosaic image augmentation is used at the input end. Multiple images are randomly scaled and cropped, then randomly distributed and stitched together, greatly enriching the dataset. In particular, random scaling increases the diversity and difficulty of the training set, helping to improve the robustness and generalization ability of the object detection model. Simultaneously, Mosaic image augmentation can reduce the risk of overfitting and improve the model's training effect; the resulting dataset has increased complexity while reducing GPU memory usage.

[0112] Backbone Network: The augmented dataset is fed into the YOLOv5 backbone neural network CSPDarknet53 for training. The backbone network first inputs the original 608×608×3 image into the Focus structure, and uses a slicing operation to transform it into a 304×304×12 feature map. Then, it undergoes a convolution operation with 32 convolutional kernels, finally becoming a 304×304×32 feature map, which is then fed into the Conv convolutional layer for convolution operation. The activation function of the convolutional layer is the SiLu activation function as shown in Equation (Ⅰ):

[0113] SiLU(x)=x·sigmoid(x) (Ⅰ)

[0114] In the SPPF module of the backbone network, the input channels are halved through a standard convolutional module, and three kernel-size 5 max pooling operations are performed. The results of the three max pooling operations are concatenated with the data that has not undergone pooling operations. Finally, the number of channels after merging is twice that of the original.

[0115] In the FPN layer of the Neck network, the feature information of the high-level layer and the feature information of the low-level layer are fused through the upsampling operation to calculate the predicted feature map. After the FPN layer, a feature pyramid is added from bottom to top, which includes two PAN structures. Through the downsampling operation, the feature information of the low-level layer and the feature information of the high-level layer are fused to output the predicted feature map.

[0116] After optimizing the output Head to ensure that the predicted bounding boxes and the ground truth bounding boxes do not intersect, the prediction results are output.

[0117] At the output end, YOLOv5 uses CIOU_Loss as the loss function for the bounding box, as shown in Equation (II):

[0118]

[0119] in:

[0120] The three training methods correspond to three different reasoning methods:

[0121] The first method is: when the target detection model is trained in the first way, the trained target detection model is used to infer the real-time classroom video stream, and the inferred video stream and classroom behavior analysis report are stored in a unified storage and data transfer platform.

[0122] The second method is as follows: When the target detection model is trained in the second way, the cloud sends inference tasks to the edge through the kubeedge management platform. The edge performs inference on the real-time classroom video stream and stores the inference video stream and classroom behavior analysis report in the unified storage and data transfer platform of the computing power network.

[0123] The third method is as follows: When the object detection model is trained in the third method, the edge server obtains the real-time classroom video stream and the trained object detection model image from the unified storage and data transfer platform. The cloud server then issues inference tasks through the kubeedge management platform, and the edge server sends the inference video stream and analysis report to the unified storage and data transfer platform.

[0124] Step 4: Use the trained object detection model to infer the real-time classroom video stream and generate a classroom behavior analysis report.

[0125] Step 1 can be deployed in two ways:

[0126] The first type is: such as Figure 4 As shown, if the user selects that the training and inference processes of the object detection model are both completed locally at the edge layer, or the training process of the object detection model is completed at the computing network while the inference process is completed locally, then an edge communication module is deployed at the edge layer to communicate with the cloud and obtain the object detection model training and inference tasks.

[0127] The second type is: such as Figure 5 As shown, if the user selects that the training and inference tasks of the object detection model are both completed in the computing network, then there is no need to deploy an edge communication module at the edge layer, and the computing network directly pushes the inference results and analysis reports to the front-end page.

[0128] Edge servers perform inference tasks: After obtaining inference data and trained model images from the data transfer platform, the cloud server distributes inference tasks through the KubeEdge management platform. The edge servers then send the inference results and analysis reports to the data transfer platform. Specific implementation process:

[0129] Create an `infer.yaml` configuration file, specifying the edge node for the inference task and related parameters. Execute `kubectl apply -f infer.yaml` on the cloud to distribute the inference task. A new pod will then be started at the edge to perform the inference task. The container ID can be viewed on the edge using the `docker ps -a` command. After the inference task completes, the inference output can be observed using the `docker logs container id -f` command. Finally, the inference video and analysis report are uploaded to the unified storage and data transfer platform of the computing network.

[0130] In step 4, the unified storage and data transfer platform pushes the obtained reasoning video stream and classroom behavior analysis report to the front end. The classroom behavior analysis report is as follows: Figure 6 As shown, the system displays course name, class, location, instructor, and number of students. It analyzes student behavior to assess classroom focus and activity, presenting this information as a graph. Each lesson's live video stream generates a corresponding visual analysis report. Long-term evaluation of a course is conducted by combining the reasoning results of each lesson with the timetable information. At the end of the semester, a comprehensive focus report is provided. The focus report matches the knowledge points to be covered in each lesson of the course syllabus with the students' focus levels for that lesson. Instructors can visually identify areas with low student engagement and adjust their syllabus and teaching methods accordingly to improve teaching quality. Furthermore, schools can use the classroom and instructor information in the focus report to conduct an overall evaluation of the instructor's teaching throughout the semester, helping to objectively assess the instructor's teaching level.

[0131] Example 5

[0132] The difference between the teaching behavior analysis method based on computing power network described in Example 4 and the following is:

[0133] This example demonstrates a teacher at Qilu University of Technology's Qingdao campus applying for teaching behavior analysis services. The specific implementation steps are as follows:

[0134] Step 1. Deployment of edge devices: Deploy the kubeedge edge core module and communication module at the Qingdao campus.

[0135] Step 2. The teacher uploads the course information, class videos, and service requirements for this semester to the unified storage and data transfer platform of the computing network center.

[0136] Step 3. The unified storage and data transfer platform calls the automatic data processing module to cut the 1-hour class video into about 5,000 high-quality images and automatically label them, and divide them into training set and test set in an 8:2 ratio.

[0137] Step 4. The computing network center schedules appropriate storage and computing nodes based on the user's geographical location and service needs. Data processed on the unified storage platform is transferred to the corresponding storage nodes for storage when the network is idle. In this example, the computing network schedules computing nodes in the Qingdao area; therefore, data is transferred to storage nodes in the Qingdao area to ensure data transmission speed during model training.

[0138] Step 5. The computing node in Qingdao begins training the object detection model using the processed training set data. After training, the final accuracy reaches 0.924, meeting the user's accuracy requirements. Then, the model and training task are distributed from the cloud, and real-time video streams are inferred and published at the edge of the Qingdao campus. A teaching behavior analysis report is generated and sent to the teacher, who can use this report to reflect on their teaching after each class. Simultaneously, the computing network saves the user's final trained model file, eliminating the need to retrain the model for subsequent class videos. Furthermore, based on the teacher's provided class schedule information and student performance in each class, a long-term evaluation of the teacher's course is conducted, resulting in a student focus analysis report at the end of the semester, allowing the teacher to adjust the syllabus accordingly.

[0139] Thus far, the system flow and operation steps of this invention have been described in detail and illustrated with multiple figures. The above descriptions are all preferred embodiments of the invention, provided for a better understanding of the system's operation, and are not intended to limit the scope of protection of this invention. Conversely, any modifications or alterations based on this invention should be included within the scope of protection of this invention.

Claims

1. A teaching behavior analysis system based on computing power networks, characterized in that, include: The cloud computing platform layer, communication network layer, edge layer, and web interface for user interaction; The cloud platform layer is used for: data annotation; Training the object detection model; storing the dataset, images, object detection model files, and the data flow between various storage nodes; The communication network layer is used for data and information exchange between the cloud platform layer and the edge layer. Edge layer, used for: inference of real-time video streams; A web-based interface for user interaction, used to provide users with services such as uploading data, selecting model training methods, viewing model training results, and receiving teaching behavior analysis reports; The computing cloud platform layer includes the kubeedge cloud core module, cloud communication module, computing resources, unified data storage and data transfer platform, and automatic data labeling module; The cloud core module is a component that interacts with the cloud. It is used to send instructions from the cloud to the edge layer and is also responsible for receiving events reported from the edge layer to the cloud. The cloud communication module acts as a network proxy for the cloud, through which all cloud traffic passes; enabling communication between the cloud core module and the edge layer. The computing resources of the computing network are used for training the target detection model; The unified data storage and data transfer platform is used to store, in object storage mode, the original video sources uploaded by university users, the data used for training the target detection model after being labeled by the automatic data labeling module, the historical and best versions of the target detection model, the inference results of the target detection model in the edge layer, and the teaching behavior analysis reports generated for users; and to transfer the data to the best storage node according to the user's needs. The automatic data annotation module is used to: segment the video information provided by the user into images, automatically annotate the behavior of students in the images, and export a training set that meets the training requirements of the target detection model; The unified data storage and data flow platform is a storage platform in the computing power network, including object storage, HDFS storage, NFS storage and parallel file storage. The unified data storage and data flow platform controls all storage nodes and enables data to flow between different computing nodes, and provides API interfaces for data access. Methods for training object detection models include: The target detection model uses YOLOv5, which includes an input, a backbone network, a neck network, and an output head. Data augmentation is performed at the input end using the Mosaic method; The backbone network performs slicing operations, inputting a 608×608×3 three-channel image into the Focus structure. After slicing, it is first transformed into a 304×304×12 feature map. Then, after convolution operations using 32 convolution kernels, a final 304×304×32 feature map is obtained. In the FPN layer of the Neck network, the feature information of the high-level layer and the feature information of the low-level layer are fused through the upsampling operation to calculate the predicted feature map. After the FPN layer, a feature pyramid is added from bottom to top, which includes two PAN structures. Through the downsampling operation, the feature information of the low-level layer and the feature information of the high-level layer are fused to output the predicted feature map. After optimizing the output Head to ensure that the predicted bounding boxes and the ground truth bounding boxes do not intersect, the prediction results are output.

2. The teaching behavior analysis system based on computing power network according to claim 1, characterized in that, The edge layer includes the kubeedge edge core module, edge communication module, and computing resources for university users; The edge core module is used to manage the components of the edge load. It is responsible for receiving and executing instructions sent from the cloud, that is, managing the entire life cycle of the edge load, and reporting the status of the edge load to the cloud in the form of events. The edge communication module acts as a network proxy for the edge layer, through which all traffic entering and leaving the edge layer passes; enabling communication between the cloud core module and the edge layer. The computing resources of the university users are their GPU devices, which are used to execute inference tasks distributed from the cloud and store the inference results in the unified data storage and data transfer platform. The cloud communication module and the edge communication module communicate bidirectionally via the WebSocket protocol.

3. A teaching behavior analysis method based on computing power networks, implemented through the teaching behavior analysis system based on computing power networks as described in claim 1 or 2, characterized in that, include: Step 1: Edge layer device deployment: Deploy the corresponding edge core modules and edge communication modules according to the training model method selected by the user; Step 2: Source data storage and processing: Store the original teaching video data uploaded by users into the unified storage and data transfer platform of the computing power network, and process the original teaching video data in the unified storage and data transfer platform; Step 3: Object detection model training method: The computing power network allocates suitable cloud servers or edge servers based on the user selection in Step 1, the size of the training task, and the geographical location of the user. The training data stored in the unified storage and data transfer platform in Step 2 is transferred to the most suitable server allocated by the computing power network in Step 3 to train the object detection model. There are three training methods depending on the allocated server. The object detection model is trained after the training method is determined. Step 4: Use the trained object detection model to infer the real-time classroom video stream and generate a classroom behavior analysis report.

4. The teaching behavior analysis method based on computing power network according to claim 3, characterized in that, The deployment methods in step 1 include the following two: The first option is: if the user selects that the training and inference processes of the object detection model are both completed locally at the edge layer, or the training process of the object detection model is completed at the computing network while the inference process is completed locally, then an edge communication module is deployed at the edge layer to communicate with the cloud and obtain the object detection model training and inference tasks. The second option is: if the user selects that the training and inference tasks of the object detection model are both completed on the computing network, then there is no need to deploy an edge communication module at the edge layer, and the computing network directly pushes the inference results and analysis reports to the front-end page.

5. The teaching behavior analysis method based on computing power network according to claim 3, characterized in that, Source data storage and processing, including: Step 2.1: Source data storage: Call the API interface provided by the unified storage and data transfer platform of the computing power network to store the original video source uploaded by the user and the course hours, classrooms, teachers and teaching plans in the teaching system in the unified storage and data transfer platform in the form of object storage; Step 2.2: Data Processing: Call the API interface of the automatic data labeling module to read data from the unified storage and data flow platform, import the data, automatically extract the imported original video source, and automatically label the extracted data according to the user's needs by calling the API interface provided by the automatic data labeling module, labeling the student's actions in each extracted image with corresponding labels, and dividing the labeled data into training set and test set. Step 2.3: Call the API interface of the automatic data labeling module to export the labeled data and transfer it to the unified data storage and data transfer platform of the computing power network in the form of object storage.

6. The teaching behavior analysis method based on computing power network according to claim 3, characterized in that, The three methods for training an object detection model are as follows: The first approach is: if the server matched by the computing power network for the training and inference process of the object detection model is a cloud server, then the training data is directly obtained from the unified storage and data transfer platform of the computing power network, and the object detection model is trained. The second scenario is: if the server matched by the computing power network for the training and inference process of the object detection model is an edge server, then the edge server first obtains the training data, inference data and the original object detection model image from the unified storage and data transfer platform and stores them in the local file storage system. The cloud sends training tasks to the edge server through the kubeedge management platform, and the edge server trains the object detection model. The third scenario is as follows: If the server used for training the object detection model in the computing power network is a cloud server, while the server used for model inference is an edge server, then the cloud server obtains the training data and the original object detection model image from the unified storage and data transfer platform of the computing power network, trains the object detection model with the training data, and finally generates an image of the object detection model with the best training result and sends it to the unified storage and data transfer platform.

7. The teaching behavior analysis method based on computing power network according to claim 6, characterized in that, Three different training methods correspond to three different reasoning methods: The first method is: when the target detection model is trained in the first way, the trained target detection model is used to infer the real-time classroom video stream, and the inferred video stream and classroom behavior analysis report are stored in a unified storage and data transfer platform. The second method is as follows: When the target detection model is trained in the second way, the cloud sends inference tasks to the edge through the kubeedge management platform. The edge performs inference on the real-time classroom video stream and stores the inference video stream and classroom behavior analysis report in the unified storage and data transfer platform of the computing power network. The third method is as follows: When the object detection model is trained in the third method, the edge server obtains the real-time classroom video stream and the trained object detection model image from the unified storage and data transfer platform. The cloud server then issues inference tasks through the kubeedge management platform, and the edge server sends the inference video stream and analysis report to the unified storage and data transfer platform.

8. A teaching behavior analysis method based on computing power networks according to any one of claims 3-7, characterized in that, In step 4, the unified storage and data transfer platform pushes the obtained reasoning video stream and classroom behavior analysis report to the front end. Each class's real-time video stream generates a corresponding visual analysis report, and a long-term evaluation is conducted for a course. The reasoning results of each class in the course are combined with the timetable information. As the course progresses, an overall focus report for the course is given at the end of the semester.

Images