A method and system for intelligent analysis and feedback of teachers' classroom teaching behavior

Abstract

This invention relates to the technical field of teacher classroom teaching behavior management, and discloses a method and system for intelligent analysis and feedback of teacher classroom teaching behavior. The system includes a teacher classroom teaching scene information processing module, a teacher classroom teaching behavior recognition module, and a teacher classroom teaching behavior feedback module. It uses video frame information from the teacher's classroom teaching scene or multi-view video frame information from the teacher's classroom teaching scene, combined with artificial intelligence algorithms and scientifically set different types of teacher classroom teaching behavior video frame sets, to identify the types of teacher classroom teaching behavior. This achieves efficient and standardized identification of teacher classroom teaching behavior types on the video side in teaching scenarios where video information exists independently or audio and video information coexist. It enables intelligent real-time identification of teacher classroom teaching behavior based on the audio and video scene types present in the teacher's teaching scene, improving the quality and accuracy of teacher classroom teaching behavior supervision.

Description

Technical Field

[0001] This invention relates to the technical field of teacher classroom teaching behavior management, specifically to a method and system for intelligent analysis and feedback of teacher classroom teaching behavior. Background Technology

[0002] Classroom teaching behavior refers to the practical methods by which teachers conduct teaching activities in the classroom according to specific teaching models. Its core characteristics include directionality, operability, and flexibility. It is generally defined as "all behaviors of teachers that induce, maintain, and promote student learning." This term encompasses five aspects: directionality emphasizes design around teaching objectives and requires selecting teaching models based on context; operability is reflected in transforming teaching theories into standardized behavioral frameworks, achieved through explicit means such as professional language and tools; integrity requires maintaining a structural unity between theory and practice; stability refers to its reflection of general teaching laws but is constrained by social conditions; and flexibility requires dynamic adjustment based on the characteristics of the subject and the actual situation of teachers and students. The theoretical system of classroom teaching behavior originates from the systematic research of teaching models, forming standardized characteristics through the extraction of teaching elements, and continuously evolves in contemporary educational practice to adapt to diversified teaching needs. In modern education, dynamic identification of teachers' classroom teaching behavior based on AI technology helps improve teachers' teaching level. However, existing teacher classroom teaching behavior identification systems cannot dynamically and intelligently identify teaching behaviors based on the audio and video scene types present in the teaching environment, nor can they provide accurate and visual feedback on teachers' classroom teaching behaviors.

[0003] Chinese invention patent application CN117114932A, published on November 24, 2023, discloses a teaching behavior analysis system and method based on a computing network. It establishes a computing network cloud platform layer, a communication network layer, an edge layer, and a web interface for user interaction. The computing network cloud platform layer is used for data annotation, training of the target detection model, storage of datasets, images, target detection model files, and data flow between various storage nodes. The communication network layer is used for data and information interaction between the computing network cloud platform layer and the edge layer. The edge layer is used for inference on real-time video streams. The web interface provides users with services such as uploading data, selecting model training methods, viewing model training results, and receiving teaching behavior analysis reports. However, the above technical solution cannot achieve intelligent recognition of teachers' classroom teaching behavior based on audio and video information from the classroom teaching scene. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] To address the shortcomings of existing teacher classroom teaching behavior recognition systems, which cannot dynamically and accurately identify teaching behaviors based on the types of audio and video scenarios present in the teaching environment, nor can they provide precise and visual feedback on teacher classroom teaching behaviors, this system aims to achieve the following: real-time collection of audio-visual information from the teacher classroom; scientific determination of the types of audio and video scenarios present in the teacher classroom; accurate acquisition of audio and video feature information from the teacher classroom; flexible collection of multi-view video feature information from the teacher classroom; intelligent classification and identification of teacher classroom teaching behaviors based on the types of audio and video scenarios present in the teacher classroom; and precise and visual feedback on teacher classroom teaching behaviors.

[0006] (II) Technical Solution

[0007] This invention is achieved through the following technical solution: a method for intelligent analysis and feedback of teachers' classroom teaching behavior, the method comprising the following steps:

[0008] S1. Collect audio-visual information from the teacher's classroom teaching and decompose and process the audio-visual information from the teacher's classroom teaching to construct audio information and video information from the teacher's classroom teaching.

[0009] S2. Based on the audio information of the teacher's classroom teaching, determine the existence status of the audio information of the teacher's classroom teaching and generate audio existence determination information of the teacher's classroom teaching; if it does not exist, obtain the multi-view video information of the teacher's classroom teaching and directly execute S4.

[0010] S3. When present, the audio text information of the teacher's classroom teaching scene is translated based on the audio information of the teacher's classroom teaching scene to generate audio text information of the teacher's classroom teaching scene. This audio text information is then combined with standard audio text information of different types of teacher classroom teaching behaviors to perform teacher classroom teaching behavior type identification processing to generate audio side identification information of teacher classroom teaching behavior.

[0011] S4. Based on the video information of the teacher's classroom teaching scene or the video information of the teacher's classroom teaching from multiple perspectives, extract and process the video image information of the teacher's classroom teaching scene to generate video frame information of the teacher's classroom teaching scene or video frame information of the teacher's classroom teaching from multiple perspectives. Combine this with standard video frame sets of different types of teacher classroom teaching behaviors to perform teacher classroom teaching behavior type recognition processing to generate video-side recognition information of teacher classroom teaching behavior or video-side recognition information of teacher classroom teaching behavior from multiple perspectives.

[0012] S5. Based on the audio-visual recognition information of the teacher's classroom teaching behavior and the video frame information of the teacher's classroom teaching scene, perform abnormal detection processing of the audio-visual recognition of the teacher's classroom teaching behavior, and generate abnormal detection information of the audio-visual recognition of the teacher's classroom teaching behavior. If they are inconsistent, repeat S1 to S5 until the detection results are consistent.

[0013] S6. When the audio information is not present, cluster the teacher classroom teaching behavior identification results based on the video-side identification information of the teacher classroom teaching behavior or the multi-view video-side identification information of the teacher classroom teaching behavior to generate summary information of teacher classroom teaching behavior or summary information of teacher classroom teaching behavior from multiple perspectives.

[0014] S7. Construct a real-time monitoring system for teachers' classroom teaching behavior and push the results of the real-time monitoring of teachers' classroom teaching behavior to assignments.

[0015] Preferably, the steps for collecting and decomposing the audio-visual information of the teacher's classroom teaching to construct the audio and video information of the teacher's classroom teaching are as follows:

[0016] S11. Collect audio-visual information of the teacher giving a lecture on the teacher's podium through a camera online, and generate audio-visual information of the teacher's classroom teaching.

[0017] S12. Using audio and video editing software, the video data and audio data in the teacher's classroom teaching audio-visual information are decomposed and separated, and the teacher's classroom teaching audio information and teacher's classroom teaching video information are generated. The audio and video editing software includes any one of Audacity, Adobe Audition, and Wondershare Filmora.

[0018] Preferably, the existence status of the audio information in the teacher's classroom teaching is determined based on the audio information of the teacher's classroom teaching, and the existence status information of the teacher's classroom teaching audio is generated; when it does not exist, after obtaining the multi-view video information of the teacher's classroom teaching, the operation steps of S4 are directly executed as follows:

[0019] S21. Use a breadth-first search algorithm to perform audio data search processing on the audio information of the teacher's classroom teaching, and generate a judgment information on the existence of the teacher's classroom teaching audio based on the audio data search processing results.

[0020] When audio data is found, it indicates that there is teacher audio information in the collected audio-visual data of the teacher's classroom teaching scene. Then, the existence judgment information of the teacher's classroom teaching scene audio is output as "existence".

[0021] If no audio data is found, it means that there is no teacher audio information in the collected audio-visual data of the teacher's classroom teaching. Therefore, the output message indicating the presence of audio in the teacher's classroom teaching is "not found". The system also collects video information of the teacher lecturing from different spatial perspectives using a cloud camera, generating a multi-view video information set of the teacher's classroom teaching. And in the multi-perspective live video information collection of teachers' classroom teaching After the data collection is complete, proceed directly to step S4; where ; Indicates the number of collections Multi-perspective live video information of teachers' classroom teaching from various teaching perspectives. This represents the maximum number of teaching perspective types; the teaching perspective types include the teaching perspective from the front of the teacher, the teaching perspective from the back, the teaching perspective from above, the teaching perspective from the left, and the teaching perspective from the right.

[0022] Preferably, when the condition exists, the audio-text information of the teacher's classroom teaching scene is translated based on the audio information of the teacher's classroom teaching scene to generate audio-text information of the teacher's classroom teaching scene. This audio-text information is then combined with standard audio-text information of different types of teacher classroom teaching behaviors for teacher classroom teaching behavior type identification processing. The operation steps for generating audio-side identification information of teacher classroom teaching behavior are as follows:

[0023] S31. When the audio of the teacher's classroom teaching is deemed to exist, the audio of the teacher's classroom teaching is translated into text using voice translation software, and the audio text of the teacher's classroom teaching is generated; the voice translation software includes any one of Intelligent Translator, Microsoft Translator, and Google Translate;

[0024] S32. Establish audio-text information sets of classroom teaching behaviors of different types of teachers. , ;in Indicates the first Audio text information of different types of teacher classroom teaching behaviors corresponding to various teacher classroom teaching behaviors. This represents the maximum number of teacher classroom teaching behavior types; teacher classroom teaching behavior types include blackboard writing, multimedia presentations, teacher-student Q&A, student observation, and physical demonstrations; the audio-text information of different types of teacher classroom teaching behaviors represents the standard audio-text information set for different types of teacher classroom teaching behaviors.

[0025] S33. Using the KMP search algorithm, the audio-text information of the teacher's classroom teaching scene is compared with the audio-text information sets of different types of teacher classroom teaching behaviors. Audio text information of different types of teacher classroom teaching behaviors described in the article Perform audio-text character matching to search for audio-text information of different types of teacher classroom teaching behaviors that match the audio-text information of the teacher's classroom teaching scene. The corresponding text information of teachers' classroom teaching behavior types is used to generate audio recognition information of teachers' classroom teaching behavior through data identification.

[0026] Preferably, the steps for extracting and processing video image information of the teacher's classroom teaching scene based on the teacher's classroom teaching scene video information or the teacher's classroom teaching multi-view scene video information to generate teacher's classroom teaching scene video frame information or teacher's classroom teaching multi-view scene video frame information, and performing teacher classroom teaching behavior type recognition processing together with standard video frame sets of different types of teacher classroom teaching behaviors to generate teacher classroom teaching behavior video side recognition information or teacher classroom teaching behavior multi-view video side recognition information are as follows:

[0027] S41. Using audio and video editing software, respectively process the generated teacher classroom teaching scene video information or the teacher classroom multi-view scene video information set. The multi-perspective live video information of teachers' classroom teaching described in the article Extract and process video image information from the teacher's classroom teaching session, and generate a set of video frame information from the teacher's classroom teaching session. Or a matrix of multi-perspective live video frames from teachers' classroom teaching ;in , ;in Indicates the first Video frame information of classroom teaching from different teaching perspectives. This represents the maximum number of video frames in a teacher's classroom teaching session; where... , Indicates the first A collection of multi-perspective video frames from classroom teaching corresponding to various teaching perspective types, among which... , ,in This represents the set of multi-view live video frames of the teacher's classroom teaching. The Middle A series of video frames showing teachers' classroom teaching from multiple perspectives. This represents the set of multi-view live video frames of the teacher's classroom teaching. The multi-perspective live video frame information of teachers' classroom teaching described in the article The maximum number;

[0028] S42. Establish a video frame set matrix of classroom teaching behaviors of different types of teachers. ,in Indicates the first A collection of video frames depicting different types of teacher classroom teaching behaviors, corresponding to various teacher classroom teaching behavior types. , ,in This represents the video frame set representing the different types of teacher classroom teaching behaviors. The Middle Video frames showcasing different types of teachers' classroom teaching behaviors. This represents the video frame set representing the different types of teacher classroom teaching behaviors. Video frame information of different types of teachers' classroom teaching behaviors described in the article The maximum number;

[0029] S43. Collect the video frame information of the teacher's classroom teaching scene. The video frame information of the teacher's classroom teaching scene described in the article Or the aforementioned multi-perspective live video frame information set matrix of teacher classroom teaching The internal multi-perspective live video frame information set of teacher classroom teaching The multi-perspective live video frame information of teachers' classroom teaching described in the article The video frame set matrix of different types of teachers' classroom teaching behaviors The internal video frame set of different types of teacher classroom teaching behaviors Video frame information of different types of teachers' classroom teaching behaviors described in the article Perform video image feature matching to search for a set of video frames that match the teacher's classroom teaching scene. Or the aforementioned multi-perspective live video frame information set of teacher classroom teaching Matching video frame sets of different types of teacher classroom teaching behaviors The corresponding textual information on teacher classroom teaching behaviors is used to generate video-based identification information or multi-view video-based identification information sets of teacher classroom teaching behaviors after data identification. The process generates video-side recognition information of the teacher's classroom teaching behavior or a set of multi-view video-side recognition information of the teacher's classroom teaching behavior. The specific operating steps are as follows:

[0030] S431. Initialization phase: Update the maximum number of iterations T and the matrix of video frame sets for different types of teacher classroom teaching behaviors. The position of the teaching behavior recognition raccoon population is randomly initialized and updated within the optimization space. This population is formed by the aggregation of individual teaching behavior recognition raccoons. The encoding of teacher classroom teaching behavior video-side recognition information or multi-view video-side recognition information is defined as the teaching behavior recognition raccoon individual within the teaching behavior recognition raccoon population. The position update formula for individual teaching behavior recognition raccoons within the teaching behavior recognition raccoon population is as follows: ,in Indicating teaching behavior to identify individual raccoons exist The video frame set matrix of different types of teacher classroom teaching behaviors described in the dimension The location of the search space; The video frame set matrix of the different types of teachers' classroom teaching behaviors The upper boundary of the optimization space. The video frame set matrix of the different types of teachers' classroom teaching behaviors The lower boundary of the optimization space, where r is a random number taking values ​​in the interval [0,1].

[0031] S432, Hunting and Attack Phase, in the video frame set matrix of different types of teacher classroom teaching behaviors In the optimization space, teaching behavior recognition is performed on raccoon populations. Individual raccoons are identified through simulated attacks and video frame information sets related to classroom teaching. Or the aforementioned multi-perspective live video frame information set of teacher classroom teaching Matching video frame sets of different types of teacher classroom teaching behaviors iguana strategies for hunting and the information set of video frames from the teacher's classroom teaching. Or the aforementioned multi-perspective live video frame information set of teacher classroom teaching Matching video frame sets of different types of teacher classroom teaching behaviors Iguana; Teaching behavior recognition: Raccoon individual climbs tree to search for video frame information related to the teacher's classroom teaching scene. Or the aforementioned multi-perspective live video frame information set of teacher classroom teaching Matching video frame sets of different types of teacher classroom teaching behaviors Iguana; Other teaching behavior recognition: Raccoon waits on the ground until the video frame set of the different types of teacher classroom teaching behaviors is obtained. The iguana fell to the ground; the video frame collection depicting different types of teacher classroom teaching behaviors. After the iguana landed, the teaching behavior identification system recognized raccoons attacking and hunting different types of teachers' classroom teaching behaviors, using video frames. Iguana; The algorithm design assumes that the optimal location of the raccoon member for identifying teaching behaviors in the raccoon population is the video frame set of the different types of teacher classroom teaching behaviors. The location of the iguana, assuming half of the video frame sets of the different types of teacher classroom teaching behaviors. The iguana climbs the tree, while the other half consists of video frames depicting different types of teacher classroom teaching behaviors. The iguana fell to the ground; a collection of video frames depicting different types of teacher classroom teaching behaviors as the iguana climbed up the tree. iguanas in the video frame set matrix of different types of teacher classroom teaching behaviors The mathematical simulation formula for the position in the optimization space is as follows: ,in This represents a set of video frames depicting different types of teacher classroom teaching behaviors described on the tree. iguanas in The video frame set matrix of different types of teacher classroom teaching behaviors described in the dimension The updated position in the optimization space. Represents tree-based teaching behavior to identify individual raccoons. exist The video frame set matrix of different types of teacher classroom teaching behaviors described in the dimension The updated position in the optimization space. This represents a set of video frames depicting different types of teacher classroom teaching behaviors described on the tree. iguanas in The video frame set matrix of different types of teacher classroom teaching behaviors described in the dimension The original position in the search space. It is a random integer taking values ​​in the range [0,1].

[0032] Video frame sets of different types of teachers' classroom teaching behaviors After the iguana fell to the ground, the video frame collection of different types of teacher classroom teaching behaviors was used. Iguanas were placed in a matrix of video frames depicting different types of teacher classroom teaching behaviors. A random location within the optimization space; a set of video frames simulating different types of teacher classroom teaching behaviors on the ground based on the random location. iguanas in the video frame set matrix of different types of teacher classroom teaching behaviors Moving position within the optimization space; the video frame set of different types of teacher classroom teaching behaviors The formula for simulating the position movement is: ,in This represents a set of video frames depicting different types of teacher classroom teaching behaviors described on the ground. iguanas in The video frame set matrix of different types of teacher classroom teaching behaviors described in the dimension The updated position in the optimization space. This indicates ground-based teaching behavior identification of individual raccoons. exist The video frame set matrix of different types of teacher classroom teaching behaviors described in the dimension The updated position in the optimization space. This represents a set of video frames depicting different types of teacher classroom teaching behaviors described on the ground. iguanas in The video frame set matrix of different types of teacher classroom teaching behaviors described in the dimension The original position in the search space; This represents a set of video frames depicting different types of teacher classroom teaching behaviors described on the ground. iguanas in The video frame set matrix of different types of teacher classroom teaching behaviors described in the dimension The range of update positions in the optimization space; This indicates that ground-based teaching behavior identification of individual raccoons is... The video frame set matrix of different types of teacher classroom teaching behaviors described in the dimension The range of update positions in the optimization space;

[0033] S433, the escape from predators stage, in the video frame set matrix of different types of teacher classroom teaching behaviors Teaching behavior recognition in the optimization space; teaching behavior recognition of individual raccoons in a raccoon population by simulating encounters with the teacher in a classroom teaching scene video frame information set. Or the aforementioned multi-perspective live video frame information set of teacher classroom teaching Matching video frame sets of different types of teacher classroom teaching behaviors Predator and escape from video frame information set of the teacher's classroom teaching scene Or the aforementioned multi-perspective live video frame information set of teacher classroom teaching Matching video frame sets of different types of teacher classroom teaching behaviors Predator strategies are used to search for video frame information related to the teacher's classroom teaching. Or the aforementioned multi-perspective live video frame information set of teacher classroom teaching Matching video frame sets of different types of teacher classroom teaching behaviors ; when predators are in the different types of teacher classroom teaching behavior video frame set matrix When attacking individual raccoon instances of teaching behavior recognition within the optimization space, the teaching behavior recognition raccoon instances are located in the video frame set matrix of different types of teacher classroom teaching behaviors. The raccoon escapes its current dangerous position and reaches a new safe position within the optimization space; the teaching behavior recognition simulation formula for the position update process of an individual raccoon escaping from a predator is as follows: ,in Indicates that predators are The video frame set matrix of different types of teacher classroom teaching behaviors described in the dimension The updated location during the process of identifying individual raccoons in the optimization space of attacking ground teaching behaviors. Indicating teaching behavior to identify individual raccoons exist The video frame set matrix of different types of teacher classroom teaching behaviors described in the dimension The updated position during the process of escaping the predator in the optimization space; and They represent the first The video frame set matrix of different types of teachers' classroom teaching behavior after the number of iterations The upper and lower boundaries of the optimization space;

[0034] S434. When the algorithm reaches the maximum number of iterations, the output is the set of video frame information related to the teacher's classroom teaching. Or the aforementioned multi-perspective live video frame information set of teacher classroom teaching Matching video frame sets of different types of teacher classroom teaching behaviors Otherwise, repeat steps S432 to S434 until the maximum number of iterations is reached;

[0035] S435. Combine the information set of video frames from the teacher's classroom teaching scene output in step S434 with the information set of video frames from the teacher's classroom teaching scene. Or the aforementioned multi-perspective live video frame information set of teacher classroom teaching Matching video frame sets of different types of teacher classroom teaching behaviors The corresponding textual information on teacher classroom teaching behaviors is used to generate video-based identification information or multi-view video-based identification information sets of teacher classroom teaching behaviors after data identification. ,in Indicates the first Multi-perspective video-based identification information of teachers' classroom teaching behaviors from various teaching perspectives.

[0036] Preferably, based on the audio-visual recognition information of the teacher's classroom teaching behavior and the video frame information of the teacher's classroom teaching scene, anomaly detection processing of the audio-visual recognition of the teacher's classroom teaching behavior is performed to generate anomaly detection information of the teacher's classroom teaching behavior. When there is inconsistency, the operation steps S1 to S5 are repeated until the detection results are consistent, as follows:

[0037] S51. Obtain the audio-side recognition information of the teacher's classroom teaching behavior and the video frame information of the teacher's classroom teaching scene;

[0038] S52. Match the audio-side recognition information of the teacher's classroom teaching behavior with the video frame information of the teacher's classroom teaching scene, and generate abnormal detection information of the teacher's classroom teaching behavior audio and video side based on the matching result of the teacher's classroom teaching behavior type keywords.

[0039] When the keyword matching for the teacher's classroom teaching behavior type is successful, it indicates that the teacher's classroom teaching behavior type identified on the audio side and the video side is consistent; then the abnormal detection information of the teacher's classroom teaching behavior audio and video sides is output as consistent.

[0040] If the keyword for the teacher's classroom teaching behavior type fails to match, it indicates that the teacher's classroom teaching behavior type identified by the audio side and the video side is inconsistent. In this case, the abnormal detection information of the teacher's classroom teaching behavior audio and video side is output as inconsistent. At this time, S1 to S5 are repeated until the abnormal detection information of the teacher's classroom teaching behavior audio and video side is consistent.

[0041] Preferably, the steps for clustering the teacher classroom teaching behavior recognition results based on the teacher classroom teaching behavior video-side recognition information or the teacher classroom teaching behavior multi-view video-side recognition information to generate teacher classroom teaching behavior summary information or teacher classroom teaching behavior multi-view summary information when there is consistency or when the audio information is absent are as follows:

[0042] S61. When the abnormal detection information of the teacher's classroom teaching behavior audio and video is consistent, or when the existence judgment information of the teacher's classroom teaching scene audio is absent, a clustering search algorithm is used to identify the teacher's classroom teaching behavior video or the teacher's classroom teaching behavior multi-view video identification information set. The multi-view video-based recognition information of teachers' classroom teaching behavior described in the article A keyword clustering search was conducted to identify the most frequently occurring keywords related to teachers' classroom teaching behaviors. This search was then used to construct a summary of teachers' classroom teaching behaviors or a summary of their classroom teaching behaviors from multiple perspectives.

[0043] Preferably, the steps for constructing real-time monitoring information on teachers' classroom teaching behavior and pushing the results of the real-time monitoring of teachers' classroom teaching behavior to assignments are as follows:

[0044] S71. The teacher classroom teaching scene audio-visual information, the teacher classroom teaching behavior summary information or the teacher classroom teaching multi-view scene video information and the teacher classroom teaching behavior multi-view summary information are used to construct real-time monitoring information of teacher classroom teaching behavior by data identification.

[0045] S72. The real-time monitoring information of the teacher's classroom teaching behavior is pushed online to the teaching supervision platform through the Internet of Things communication network, and the real-time monitoring information of the teacher's classroom teaching behavior is displayed and output on the display screen to execute the task of pushing the real-time monitoring results of the teacher's classroom teaching behavior.

[0046] A teacher classroom teaching behavior intelligent analysis and feedback system is provided to implement the teacher classroom teaching behavior intelligent analysis and feedback method. The system includes a teacher classroom teaching scene information processing module, a teacher classroom teaching behavior recognition module, and a teacher classroom teaching behavior feedback module.

[0047] The teacher classroom teaching scene information processing module includes a teacher classroom teaching scene audio-visual information acquisition unit, a teacher classroom teaching scene audio-visual information decomposition unit, a teacher classroom teaching scene audio information judgment unit, a teacher classroom teaching multi-view scene video information acquisition unit, a teacher classroom teaching scene audio text information translation unit, a teacher classroom teaching scene audio text information storage unit for different types of teacher classroom teaching behavior, and a teacher classroom teaching behavior audio side recognition unit.

[0048] The teacher classroom teaching scene audiovisual information acquisition unit collects audiovisual information from the teacher classroom teaching scene through a camera; the teacher classroom teaching scene audiovisual information decomposition unit decomposes the audiovisual information from the teacher classroom teaching scene based on the audiovisual information and in conjunction with audio and video editing software to construct the teacher classroom teaching scene audio information and teacher classroom teaching scene video information; the teacher classroom teaching scene audio information judgment unit judges the existence status of the teacher classroom teaching scene audio information based on the teacher classroom teaching scene audio information to generate teacher classroom teaching scene audio existence judgment information; the teacher classroom teaching multi-view scene video information acquisition unit collects teacher classroom teaching scene video information through a cloud camera. The system includes: multi-view live video information of classroom teaching; a teacher classroom teaching audio-text information translation unit that translates the teacher classroom teaching audio-text information using the teacher classroom teaching audio-text information and voice translation software to generate teacher classroom teaching audio-text information; a different type of teacher classroom teaching behavior audio-text information storage unit for storing different types of teacher classroom teaching behavior audio-text information; and a teacher classroom teaching behavior audio-side recognition unit that performs teacher classroom teaching behavior type recognition processing based on the teacher classroom teaching audio-text information and standard audio-text information of different types of teacher classroom teaching behavior to generate teacher classroom teaching behavior audio-side recognition information.

[0049] The teacher classroom teaching behavior recognition module includes a teacher classroom teaching video frame processing unit, a storage unit for video frames of different types of teacher classroom teaching behaviors, a teacher classroom teaching behavior video side recognition unit, a teacher classroom teaching behavior audio and video side recognition anomaly detection unit, and a teacher classroom teaching behavior recognition result clustering unit.

[0050] The teacher classroom teaching video frame processing unit extracts and processes video image information from the teacher classroom teaching video or the teacher classroom teaching multi-view video, generating teacher classroom teaching video frame information or teacher classroom teaching multi-view video frame information. The different types of teacher classroom teaching behavior video frame set storage unit stores different types of teacher classroom teaching behavior video frame sets. The teacher classroom teaching behavior video side recognition unit performs teacher classroom teaching behavior type recognition processing based on the teacher classroom teaching video frame information or teacher classroom teaching multi-view video frame information and standard video frame sets of different types of teacher classroom teaching behaviors, generating... The system includes: video-based identification information of teacher classroom teaching behavior or multi-view video-based identification information of teacher classroom teaching behavior; an anomaly detection unit for audio-visual identification of teacher classroom teaching behavior, which performs anomaly detection processing on the audio-visual identification information of teacher classroom teaching behavior and the video frame information of the teacher classroom teaching scene, to generate anomaly detection information on the audio-visual identification of teacher classroom teaching behavior; and a clustering unit for teacher classroom teaching behavior identification results, which performs clustering processing on the video-based identification information of teacher classroom teaching behavior or multi-view video-based identification information of teacher classroom teaching behavior, to generate summary information on teacher classroom teaching behavior or summary information on teacher classroom teaching behavior from multiple perspectives.

[0051] The teacher classroom teaching behavior feedback module includes a teacher classroom teaching behavior real-time monitoring information construction unit and a teacher classroom teaching behavior real-time monitoring result push unit.

[0052] The real-time monitoring information construction unit for teachers' classroom teaching behavior constructs real-time monitoring information for teachers' classroom teaching behavior based on the audio-visual information of teachers' classroom teaching, the summary information of teachers' classroom teaching behavior, or the multi-view video information of teachers' classroom teaching and the summary information of teachers' classroom teaching behavior from multiple perspectives, combined with data processing; the real-time monitoring result push unit for teachers' classroom teaching behavior executes the task of pushing the real-time monitoring results of teachers' classroom teaching behavior based on the real-time monitoring information of teachers' classroom teaching behavior and in conjunction with the teaching supervision platform and the display screen.

[0053] (III) Beneficial Effects

[0054] This invention provides a method and system for intelligent analysis and feedback of teachers' classroom teaching behavior. It has the following beneficial effects:

[0055] I. By dynamically and accurately collecting audio and video features from the classroom teaching scene, and scientifically determining the teaching scenario type—whether the video information exists alone or the audio and video information coexists—based on the audio data, this system enables multi-method identification of teacher classroom teaching behavior based on the audio and video scene classification. Cloud cameras are used to collect multi-view video information from different perspectives for teaching scenarios where the video information exists alone, improving the accuracy of teacher classroom behavior recognition. Furthermore, voice translation software efficiently translates the audio text information from the classroom teaching scene, and this is combined with the intelligent search algorithm and standard audio text information of different types of teacher classroom teaching behavior stored in big data for efficient identification of teacher classroom teaching behavior types. This achieves intelligent identification of audio-side teacher classroom teaching behavior types in teaching scenarios where audio and video information coexist.

[0056] Second, by combining artificial intelligence algorithms with standard video frame sets of different types of teacher classroom teaching behaviors based on video frame information from the teacher's classroom teaching scene or multi-view video frame information from the teacher's classroom teaching scene, this system performs efficient and standardized identification of teacher classroom teaching behavior types on the video side in teaching scenarios where video information exists alone or audio and video information coexist. Based on the audio-side identification information and video frame information of the teacher's classroom teaching behavior, this system performs anomaly detection processing on the audio and video side of the teacher's classroom teaching behavior, enabling intelligent detection of anomalies in the audio and video side of the teacher's classroom teaching behavior type identification in teaching scenarios where audio and video information coexist, thus improving the reliability of teacher classroom teaching behavior identification in such scenarios. This system also enables the scientific aggregation of teacher classroom teaching behavior identification results in teaching scenarios where video information exists alone or audio and video information coexist. Finally, it enables intelligent real-time identification of teacher classroom teaching behavior based on the audio and video scene types present in the teacher's teaching scene, improving the quality and accuracy of teacher classroom teaching behavior monitoring.

[0057] Third, by scientifically constructing real-time monitoring information of teachers' classroom teaching behavior based on the audio-visual information of teachers' classroom teaching, the summary information of teachers' classroom teaching behavior, or the multi-perspective on-site video information of teachers' classroom teaching and the summary information of teachers' classroom teaching behavior from multiple perspectives, and combining data processing, we can achieve the dynamic and intuitive feedback of the results of teachers' classroom teaching behavior identification. At the same time, in conjunction with the teaching supervision platform and the display screen, we can efficiently and visually push the real-time monitoring results of teachers' classroom teaching behavior to homework, so as to achieve dynamic and intuitive feedback of the results of teachers' classroom teaching behavior identification. Attached Figure Description

[0058] Figure 1 This invention provides a schematic diagram of a module for an intelligent analysis and feedback system for teachers' classroom teaching behavior.

[0059] Figure 2 The flowchart of an intelligent analysis and feedback method for teachers' classroom teaching behavior provided by the present invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] An example of a method and system for intelligent analysis and feedback of teachers' classroom teaching behavior is as follows:

[0062] Example 1:

[0063] Please see Figure 1 - Figure 2 A method for intelligent analysis and feedback of teachers' classroom teaching behavior, comprising the following steps:

[0064] S1. Collect audio-visual information from the teacher's classroom teaching and decompose and process the audio-visual information from the teacher's classroom teaching to construct audio information and video information from the teacher's classroom teaching.

[0065] S2. Based on the audio information of the teacher's classroom teaching, determine the existence status of the audio information in the teacher's classroom teaching and generate the audio existence determination information of the teacher's classroom teaching; if it does not exist, obtain the multi-view video information of the teacher's classroom teaching and directly execute S4.

[0066] S3. When present, translate the audio text information of the teacher's classroom teaching scene based on the audio information of the teacher's classroom teaching scene, generate the audio text information of the teacher's classroom teaching scene, and perform teacher classroom teaching behavior type identification processing together with the standard audio text information of different types of teacher classroom teaching behavior to generate teacher classroom teaching behavior audio side identification information.

[0067] S4. Based on the video information of the teacher's classroom teaching or the video information of the teacher's classroom teaching from multiple perspectives, extract and process the video image information of the teacher's classroom teaching, generate video frame information of the teacher's classroom teaching or video frame information of the teacher's classroom teaching from multiple perspectives, and combine it with the standard video frame set of different types of teacher classroom teaching behaviors to perform teacher classroom teaching behavior type recognition processing, and generate teacher classroom teaching behavior video side recognition information or teacher classroom teaching behavior multi-view video side recognition information.

[0068] S5. Based on the audio-visual recognition information of the teacher's classroom teaching behavior and the video frame information of the teacher's classroom teaching scene, perform abnormal detection processing of the audio-visual recognition of the teacher's classroom teaching behavior, and generate abnormal detection information of the audio-visual recognition of the teacher's classroom teaching behavior. If they are inconsistent, repeat S1 to S5 until the detection results are consistent.

[0069] S6. When consistent or when audio information is not available, cluster the teacher classroom teaching behavior recognition results based on the teacher classroom teaching behavior video side recognition information or the teacher classroom teaching behavior multi-view video side recognition information to generate teacher classroom teaching behavior summary information or teacher classroom teaching behavior multi-view summary information.

[0070] S7. Construct a real-time monitoring system for teachers' classroom teaching behavior and push the results of the real-time monitoring of teachers' classroom teaching behavior to assignments.

[0071] For further details, please refer to Figure 1 - Figure 2 The steps for collecting and processing audio-visual information from teachers' classroom teaching to construct audio and video information of teachers' classroom teaching are as follows:

[0072] S11. Collect audio-visual information of the teacher giving a lecture on the teacher's podium through a camera online, and generate audio-visual information of the teacher's classroom teaching.

[0073] S12. Use audio and video editing software to decompose and separate the video and audio data in the audio and video data of the teacher's classroom teaching, and generate the teacher's classroom teaching audio information and teacher's classroom teaching video information. The audio and video editing software includes any one of Audacity, Adobe Audition, and Wondershare Filmora.

[0074] Based on the audio information from the teacher's classroom teaching, the system determines the presence of the audio information and generates presence information. If the audio is not present, the system acquires multi-view video information of the teacher's classroom teaching and directly executes the following steps in S4:

[0075] S21. Use a breadth-first search algorithm to perform audio data search processing on the audio information of the teacher's classroom teaching, and generate a judgment information on the existence of the teacher's classroom teaching audio based on the audio data search processing results.

[0076] When audio data is found, it means that there is teacher audio information in the collected audio-visual data of the teacher's classroom teaching scene. Then, the output message "existence of teacher's classroom teaching scene audio" is output.

[0077] If no audio data is found, it means that there is no teacher audio information in the collected audio-visual data of the teacher's classroom teaching. In this case, the output message "No audio exists" is output. The system also collects video information of the teacher lecturing from different spatial perspectives using cloud cameras, generating a multi-view video information set of the teacher's classroom teaching. In addition to the collection of live video information from multiple perspectives of teachers' classroom teaching After the data collection is complete, proceed directly to step S4; where ; Indicates the number of collections Multi-perspective live video information of teachers' classroom teaching from various teaching perspectives. This represents the maximum number of teaching perspective types; teaching perspective types include front teaching perspective, back teaching perspective, top teaching perspective, left teaching perspective, and right teaching perspective.

[0078] When present, the audio-text information of the teacher's classroom teaching is translated based on the audio information of the classroom teaching scene, generating audio-text information of the teacher's classroom teaching scene. This audio-text information is then combined with standard audio-text information of different types of teacher classroom teaching behaviors for teacher classroom teaching behavior type identification processing. The operation steps for generating audio-side identification information of teacher classroom teaching behaviors are as follows:

[0079] S31. When the audio of the teacher's classroom teaching is deemed to exist, the audio-text information of the teacher's classroom teaching is translated using voice translation software, and the audio-text information of the teacher's classroom teaching is generated; the voice translation software includes any one of Intelligent Translator, Microsoft Translator, and Google Translate;

[0080] S32. Establish audio-text information sets of classroom teaching behaviors of different types of teachers. , ;in Indicates the first Audio text information of different types of teacher classroom teaching behaviors corresponding to various teacher classroom teaching behaviors. This represents the maximum number of types of teacher classroom teaching behaviors; these types include blackboard writing, multimedia presentations, teacher-student Q&A, classroom observation, and physical demonstrations; the audio-text information for different types of teacher classroom teaching behaviors represents the standard audio-text information set for different types of teacher classroom teaching behaviors.

[0081] S33. Use the KMP search algorithm to combine the audio and text information of teachers' classroom teaching with audio and text information sets of different types of teachers' classroom teaching behaviors. Audio text information of classroom teaching behaviors of different types of teachers Perform audio-text character matching to search for different types of audio-text information about teachers' classroom teaching behaviors that match the audio-text information of teachers' classroom teaching. The corresponding text information of teachers' classroom teaching behavior types is used to generate audio recognition information of teachers' classroom teaching behavior through data identification.

[0082] By cooperating with the audiovisual information acquisition unit, audiovisual information decomposition unit, and audio information judgment unit of the teacher's classroom teaching scene, the system achieves dynamic and accurate acquisition of audio and video features of the teacher's classroom teaching scene. Simultaneously, based on the audio data and intelligent search algorithms, it scientifically determines the teaching scenario type where video information exists independently or audio and video information coexists, enabling multi-scheme identification of teacher classroom teaching behavior based on audio and video scene classification. The multi-view video information acquisition unit uses cloud cameras to acquire multi-view video information of the teacher's classroom teaching scene for teaching scenarios where video information exists independently, improving the accuracy of teacher classroom teaching behavior recognition. The audio-text information translation unit and the audio-side recognition unit of teacher classroom teaching behavior work together to efficiently translate the audio-text information of the teacher's classroom teaching scene using voice translation software. This is combined with intelligent search algorithms and standard audio-text information of different types of teacher classroom teaching behavior stored in big data to efficiently identify the type of teacher classroom teaching behavior, achieving intelligent identification of audio-side teacher classroom teaching behavior types in teaching scenarios where audio and video information coexist.

[0083] For further details, please refer to Figure 1 - Figure 2 The steps for extracting and processing video image information from the classroom teaching scene based on video information from teachers' actual teaching or multi-view video information from teachers' actual teaching, generating video frame information from the classroom teaching scene or multi-view video information from teachers' actual teaching, and combining this information with standard video frame sets of different types of teacher classroom teaching behaviors for teacher classroom teaching behavior type identification processing, generating video-side identification information of teacher classroom teaching behaviors or multi-view video-side identification information of teacher classroom teaching behaviors, are as follows:

[0084] S41. Use audio and video editing software to process the generated live video information of teachers' classroom teaching or the collection of live video information of teachers' classroom teaching from multiple perspectives. Multi-perspective live video information of Chinese teachers' classroom teaching Extract and process video image information from the teacher's classroom teaching session, and generate a set of video frame information from the teacher's classroom teaching session. Or a matrix of multi-perspective live video frames from teachers' classroom teaching ;in , ;in Indicates the first Video frame information of classroom teaching from different teaching perspectives. This represents the maximum number of video frames in a teacher's classroom teaching session; where... , Indicates the first A collection of multi-perspective video frames from classroom teaching corresponding to various teaching perspective types, among which... , ,in This represents a collection of video frames from multiple perspectives of classroom teaching by teachers. The Middle A series of video frames showing teachers' classroom teaching from multiple perspectives. This represents a collection of video frames from multiple perspectives of classroom teaching by teachers. Multi-perspective live video frame information of Chinese teachers' classroom teaching The maximum number;

[0085] S42. Establish a video frame set matrix of classroom teaching behaviors of different types of teachers. ,in Indicates the first A collection of video frames depicting different types of teacher classroom teaching behaviors, corresponding to various teacher classroom teaching behavior types. , ,in Video frame set representing different types of teachers' classroom teaching behaviors The Middle Video frames showcasing different types of teachers' classroom teaching behaviors. Video frame set representing different types of teachers' classroom teaching behaviors Video frame information of classroom teaching behaviors of different types of teachers The maximum number;

[0086] S43. Collect video frame information of teachers' classroom teaching. Video frame information of classroom teaching by Chinese teachers Or a matrix of multi-perspective live video frames from teachers' classroom teaching Internal collection of multi-perspective live video frame information of teachers' classroom teaching Multi-perspective live video frame information of Chinese teachers' classroom teaching Video frame set matrix of classroom teaching behaviors of different types of teachers Internal video frame collection of different types of teachers' classroom teaching behaviors Video frame information of classroom teaching behaviors of different types of teachers Perform video image feature matching to search for a set of video frames that match the teacher's classroom teaching. Or a collection of video frames from multiple perspectives of classroom teaching by teachers Matching video frames of different types of teachers' classroom teaching behaviors The corresponding textual information on teacher classroom teaching behaviors is used to generate video-based identification information or multi-view video-based identification information sets of teacher classroom teaching behaviors after data identification. The process generates video-based recognition information of teachers' classroom teaching behavior or a set of multi-view video-based recognition information of teachers' classroom teaching behavior. The specific operating steps are as follows:

[0087] S431. Initialization phase: Update the maximum number of iterations T and the video frame set matrix for different types of teachers' classroom teaching behaviors. The position of the teaching behavior recognition raccoon population is randomly initialized and updated within the optimization space. This population is formed by the aggregation of individual teaching behavior recognition raccoons. The encoding of teacher classroom teaching behavior video-side recognition information or multi-view video-side recognition information is defined as the teaching behavior recognition raccoon individual within the teaching behavior recognition raccoon population. The position update formula for individual teaching behavior recognition raccoons within the teaching behavior recognition raccoon population is as follows: ,in Indicating teaching behavior to identify individual raccoons exist Different types of teacher classroom teaching behavior video frame set matrix The location of the search space; Video frame set matrix for different types of teachers' classroom teaching behaviors The upper boundary of the optimization space. Video frame set matrix for different types of teachers' classroom teaching behaviors The lower boundary of the optimization space, where r is a random number taking values ​​in the interval [0,1].

[0088] S432, Hunting and Attack Phase, Video Frame Set Matrix of Different Types of Teacher Classroom Teaching Behaviors Teaching behavior recognition in the raccoon population within the optimization space; teaching behavior recognition of individual raccoons through simulated attacks and classroom teaching video frame information sets. Or a collection of video frames from multiple perspectives of classroom teaching by teachers Matching video frames of different types of teachers' classroom teaching behaviors iguana hunting strategies and video frame information collection of teachers' classroom teaching Or a collection of video frames from multiple perspectives of classroom teaching by teachers Matching video frames of different types of teachers' classroom teaching behaviors Iguana; Teaching behavior recognition: Raccoon individuals climbing trees to search and information from video frames of classroom teaching. Or a collection of video frames from multiple perspectives of classroom teaching by teachers Matching video frames of different types of teachers' classroom teaching behaviors Iguana; Other teaching behavior recognition: Raccoon waits on the ground until different types of teacher classroom teaching behavior video frames are collected. A collection of video frames depicting different types of classroom teaching behaviors of iguanas falling to the ground. After the iguana landed, the teaching behavior was identified, and a collection of video frames depicting different types of teachers' classroom teaching behaviors was used to identify raccoon attacks and hunting. Iguana; The algorithm design assumes that the optimal location of the raccoon member for identifying teaching behaviors in the raccoon population is a set of video frames representing different types of teachers' classroom teaching behaviors. The location of the iguana, assuming half of the different types of teachers' classroom teaching behavior video frame set iguanas climbing trees, the other half of which is a collection of video frames depicting different types of teachers' classroom teaching behaviors. iguana falls to the ground; video frame collection of different types of classroom teaching behaviors of teachers climbing trees. iguana video frame set matrix of classroom teaching behaviors of different types of teachers The mathematical simulation formula for the position in the optimization space is as follows: ,in This represents a collection of video frames depicting different types of teachers' classroom teaching behaviors on a tree. iguanas in Different types of teacher classroom teaching behavior video frame set matrix The updated position in the optimization space. Represents tree-based teaching behavior to identify individual raccoons. exist Different types of teacher classroom teaching behavior video frame set matrix The updated position in the optimization space. This represents a collection of video frames depicting different types of teachers' classroom teaching behaviors on a tree. iguanas in Different types of teacher classroom teaching behavior video frame set matrix The original position in the search space. It is a random integer taking values ​​in the range [0,1].

[0089] Video frame set of classroom teaching behaviors of different types of teachers A collection of video frames depicting different types of classroom teaching behaviors of teachers after an iguana falls to the ground. Iguanas were placed in a matrix of video frames depicting different types of teachers' classroom teaching behaviors. A random location within the optimization space; a set of video frames simulating different types of teacher classroom teaching behaviors on the ground based on the random location. iguana video frame set matrix of classroom teaching behaviors of different types of teachers Moving position within the optimization space; video frame collection of different types of teachers' classroom teaching behaviors The formula for simulating the position movement is: ,in Video frame set representing different types of teachers' classroom teaching behaviors on the ground iguanas in Different types of teacher classroom teaching behavior video frame set matrix The updated position in the optimization space. This indicates ground-based teaching behavior identification of individual raccoons. exist Different types of teacher classroom teaching behavior video frame set matrix The updated position in the optimization space. Video frame set representing different types of teachers' classroom teaching behaviors on the ground iguanas in Different types of teacher classroom teaching behavior video frame set matrix The original position in the search space; Video frame set representing different types of teachers' classroom teaching behaviors on the ground iguanas in Different types of teacher classroom teaching behavior video frame set matrix The range of update positions in the optimization space; This indicates that ground-based teaching behavior identification of individual raccoons is... Different types of teacher classroom teaching behavior video frame set matrix The range of update positions in the optimization space;

[0090] S433, the stage of escaping the predator, and the video frame set matrix of different types of teachers' classroom teaching behaviors. Teaching behavior identification in the optimization space of raccoon populations; teaching behavior identification of individual raccoons through simulated encounters with teachers in classroom teaching video frame information sets. Or a collection of video frames from multiple perspectives of classroom teaching by teachers Matching video frames of different types of teachers' classroom teaching behaviors Predator and escape and video frame information set of teacher classroom teaching Or a collection of video frames from multiple perspectives of classroom teaching by teachers Matching video frames of different types of teachers' classroom teaching behaviors Predator strategies were used to search for and extract video frame information from classroom teaching sessions. Or a collection of video frames from multiple perspectives of classroom teaching by teachers Matching video frames of different types of teachers' classroom teaching behaviors When predators are in different types of classroom teaching behavior video frame set matrix When attacking individual raccoon instances of teaching behavior identification within the optimization space, the teaching behavior identification raccoon instances are represented by a matrix of video frame sets of different types of teacher classroom teaching behaviors. The raccoon escapes its current dangerous position and reaches a new safe position within the optimization space; the teaching behavior recognition simulation formula for the position update process of an individual raccoon escaping from a predator is as follows: ,in Indicates that predators are Different types of teacher classroom teaching behavior video frame set matrix The updated location during the process of identifying individual raccoons in the optimization space of attacking ground teaching behaviors. Indicating teaching behavior to identify individual raccoons exist Different types of teacher classroom teaching behavior video frame set matrix The updated position during the process of escaping the predator in the optimization space; and They represent the first After several iterations, a video frame set matrix of different types of teachers' classroom teaching behaviors The upper and lower boundaries of the optimization space;

[0091] S434. When the algorithm reaches the maximum number of iterations, the output is a set of video frame information related to the teacher's classroom teaching. Or a collection of video frames from multiple perspectives of classroom teaching by teachers Matching video frames of different types of teachers' classroom teaching behaviors Otherwise, repeat steps S432 to S434 until the maximum number of iterations is reached;

[0092] S435. Combine the video frame information set output in step S434 with the teacher's classroom teaching scene. Or a collection of video frames from multiple perspectives of classroom teaching by teachers Matching video frames of different types of teachers' classroom teaching behaviors The corresponding textual information on teacher classroom teaching behaviors is used to generate video-based identification information or multi-view video-based identification information sets of teacher classroom teaching behaviors after data identification. ,in Indicates the first Multi-perspective video-based identification information of teachers' classroom teaching behaviors from various teaching perspectives.

[0093] Based on the audio-visual recognition information of the teacher's classroom teaching behavior and the video frame information of the teacher's classroom teaching, anomaly detection processing is performed on the audio-visual recognition of the teacher's classroom teaching behavior to generate anomaly detection information. When there is inconsistency, the operation steps S1 to S5 are repeated until the detection results are consistent, as follows:

[0094] S51. Obtain audio recognition information of teachers' classroom teaching behavior and video frame information of teachers' classroom teaching;

[0095] S52. Match the audio-side recognition information of the teacher's classroom teaching behavior with the video frame information of the teacher's classroom teaching behavior type keywords, and generate abnormal detection information of the teacher's classroom teaching behavior audio and video side based on the keyword matching results of the teacher's classroom teaching behavior type keywords.

[0096] When the keyword matching for the teacher's classroom teaching behavior type is successful, it means that the teacher's classroom teaching behavior type identified by the audio side and the video side is consistent; then the output of the abnormal detection information for the teacher's classroom teaching behavior audio and video sides is consistent.

[0097] If the keyword for the teacher's classroom teaching behavior type fails to match, it indicates that the teacher's classroom teaching behavior type identified by the audio side and the video side is inconsistent. In this case, the output of the abnormal detection information of the teacher's classroom teaching behavior audio and video side is inconsistent. At this time, S1 to S5 are repeated until the abnormal detection information of the teacher's classroom teaching behavior audio and video side is consistent.

[0098] When there is consistency or when audio information is unavailable, the following steps are taken to cluster the teacher classroom teaching behavior recognition results based on the teacher classroom teaching behavior video-side recognition information or the teacher classroom teaching behavior multi-view video-side recognition information to generate summary information of teacher classroom teaching behavior or summary information of teacher classroom teaching behavior from multiple perspectives:

[0099] S61. When the abnormal detection information of the teacher's classroom teaching behavior audio and video is consistent, or when the judgment information of the teacher's classroom teaching scene audio is not present, a clustering search algorithm is used to identify the teacher's classroom teaching behavior video or the set of teacher's classroom teaching behavior multi-view video identification information. Multi-perspective video recognition information of Chinese teachers' classroom teaching behavior A keyword clustering search was conducted to identify the most frequently occurring keywords related to teachers' classroom teaching behaviors. This search was then used to construct a summary of teachers' classroom teaching behaviors or a summary of their classroom teaching behaviors from multiple perspectives.

[0100] The teacher classroom teaching behavior video-side recognition unit identifies teacher classroom teaching behavior types by combining on-site video frame information or multi-view on-site video frame information with artificial intelligence algorithms and scientifically set standard video frame sets for different types of teacher classroom teaching behaviors. This enables efficient and standardized identification of teacher classroom teaching behavior types in teaching scenarios where video information exists independently or audio and video information coexists. The teacher classroom teaching behavior audio-visual-side recognition anomaly detection unit detects anomalies in teacher classroom teaching behavior based on audio-side recognition information and on-site video frame information. The system includes anomaly detection processing to intelligently detect anomalies in the audio and video aspects of teacher classroom teaching in scenarios where audio and video information coexist, improving the reliability of teacher classroom behavior identification in such scenarios. It also includes a teacher classroom behavior identification result clustering unit to scientifically summarize the identification results for teacher classroom behavior in scenarios where video information exists independently or where audio and video information coexist. Finally, it enables intelligent real-time identification of teacher classroom behavior based on the type of audio and video present in the teaching environment, improving the quality and accuracy of teacher classroom behavior monitoring.

[0101] For further details, please refer to Figure 1 - Figure 2 The operational steps for constructing real-time monitoring information on teachers' classroom teaching behavior and pushing the results of this monitoring to assignments are as follows:

[0102] S71. Data identification is performed on the audio-visual information of teachers’ classroom teaching, the summary information of teachers’ classroom teaching behavior, or the multi-view video information of teachers’ classroom teaching and the summary information of teachers’ classroom teaching behavior from multiple perspectives to construct real-time monitoring information of teachers’ classroom teaching behavior.

[0103] S72. Real-time monitoring information of teachers' classroom teaching behavior is pushed online to the teaching supervision platform through the Internet of Things communication network, and the real-time monitoring information of teachers' classroom teaching behavior is displayed on the display screen. The results of the real-time monitoring of teachers' classroom teaching behavior are pushed to the teaching supervision platform.

[0104] By coordinating the real-time monitoring information construction unit and the real-time monitoring result push unit for teachers' classroom teaching behavior, real-time monitoring information for teachers' classroom teaching behavior is scientifically constructed based on the audio-visual information of teachers' classroom teaching, the summary information of teachers' classroom teaching behavior, or the multi-view live video information of teachers' classroom teaching and the summary information of teachers' classroom teaching behavior from multiple perspectives, combined with data processing. At the same time, in conjunction with the teaching supervision platform and the display screen, the task of pushing the real-time monitoring results of teachers' classroom teaching behavior is carried out efficiently and visually, so as to realize the dynamic and intuitive feedback of the identification results of teachers' classroom teaching behavior.

[0105] Example 2:

[0106] Please see Figure 1 - Figure 2 A teacher classroom teaching behavior intelligent analysis and feedback system is used to implement a teacher classroom teaching behavior intelligent analysis and feedback method. The system includes a teacher classroom teaching scene information processing module, a teacher classroom teaching behavior recognition module, and a teacher classroom teaching behavior feedback module.

[0107] The teacher classroom teaching scene information processing module includes a teacher classroom teaching scene audio-visual information collection unit, a teacher classroom teaching scene audio-visual information decomposition unit, a teacher classroom teaching scene audio information judgment unit, a teacher classroom teaching multi-view scene video information acquisition unit, a teacher classroom teaching scene audio text information translation unit, a teacher classroom teaching scene audio text information storage unit, and a teacher classroom teaching behavior audio side recognition unit.

[0108] The system comprises several modules: a classroom audiovisual information acquisition unit, which collects audiovisual information from the classroom using cameras; a classroom audiovisual information decomposition unit, which processes the audiovisual information using audio and video editing software to create audio and video data; an audio information judgment unit, which determines the presence of audio data and generates audio presence status information; and a multi-view classroom video information acquisition unit, which captures video data from the classroom using cloud cameras. The system includes: multi-perspective live video information of classroom teaching; a unit for translating audio and text information of teachers' classroom teaching, which translates and processes the audio and text information of teachers' classroom teaching based on the audio and text information of teachers' classroom teaching and combines it with voice translation software to generate audio and text information of teachers' classroom teaching; a storage unit for audio and text information of different types of teachers' classroom teaching behaviors, used to store audio and text information of different types of teachers' classroom teaching behaviors; and a teacher classroom teaching behavior audio-side recognition unit, which identifies the type of teacher classroom teaching behavior based on the audio and text information of teachers' classroom teaching and the standard audio and text information of different types of teacher classroom teaching behaviors, generating audio-side recognition information of teacher classroom teaching behaviors.

[0109] The teacher classroom teaching behavior recognition module includes a teacher classroom teaching video frame processing unit, a storage unit for video frames of different types of teacher classroom teaching behaviors, a teacher classroom teaching behavior video-side recognition unit, a teacher classroom teaching behavior audio and video-side recognition anomaly detection unit, and a teacher classroom teaching behavior recognition result clustering unit.

[0110] The system includes a classroom teaching video frame processing unit, which extracts and processes video image information from the classroom teaching scene or multi-view video information to generate classroom teaching video frame information; a storage unit for video frame sets of different types of teacher classroom teaching behaviors; and a teacher classroom teaching behavior video recognition unit, which performs teacher classroom teaching behavior type recognition processing based on the classroom teaching video frame information or multi-view video frame information and standard video frame sets of different types of teacher classroom teaching behaviors, generating teaching behavior data. The system includes: a video-based identification unit for teacher classroom teaching behavior or a multi-view video-based identification unit for teacher classroom teaching behavior; an audio-visual anomaly detection unit for teacher classroom teaching behavior, which performs anomaly detection processing based on the audio-visual identification information and video frame information of the teacher classroom teaching behavior, and generates anomaly detection information for teacher classroom teaching behavior; and a clustering unit for teacher classroom teaching behavior identification results, which performs clustering processing based on the video-based identification information or the multi-view video-based identification information of teacher classroom teaching behavior, and generates summary information of teacher classroom teaching behavior or summary information of teacher classroom teaching behavior from multiple perspectives.

[0111] The teacher classroom teaching behavior feedback module includes a unit for constructing real-time monitoring information on teacher classroom teaching behavior and a unit for pushing real-time monitoring results of teacher classroom teaching behavior.

[0112] The unit for constructing real-time monitoring information on teachers' classroom teaching behavior builds real-time monitoring information on teachers' classroom teaching behavior based on audio-visual information of teachers' classroom teaching, summary information on teachers' classroom teaching behavior, or multi-view video information of teachers' classroom teaching and summary information on teachers' classroom teaching behavior from multiple perspectives, combined with data processing. The unit for pushing real-time monitoring results of teachers' classroom teaching behavior pushes the results of real-time monitoring of teachers' classroom teaching behavior based on the real-time monitoring information of teachers' classroom teaching behavior and in conjunction with the teaching supervision platform and display screen.

[0113] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent analysis and feedback of teachers' classroom teaching behavior, characterized in that, The method includes the following steps: S1. Collect audio-visual information from the teacher's classroom teaching and decompose and process the audio-visual information from the teacher's classroom teaching to construct audio information and video information from the teacher's classroom teaching. S2. Perform a status check on the audio information of the teacher's classroom teaching and generate audio presence check information; if the audio does not exist, obtain multi-view video information of the teacher's classroom teaching and execute S4 directly. S3. When present, translate the audio text information of the teacher's classroom teaching scene to generate audio text information of the teacher's classroom teaching scene. Then, together with the standard audio text information of different types of teacher classroom teaching behavior, perform teacher classroom teaching behavior type identification processing to generate audio side identification information of teacher classroom teaching behavior. S4. Extract and process video image information of the teacher's classroom teaching scene, generate video frame information of the teacher's classroom teaching scene or video frame information of the teacher's classroom teaching scene from multiple perspectives, and perform teacher classroom teaching behavior type identification processing together with standard video frame sets of different types of teacher classroom teaching behaviors to generate teacher classroom teaching behavior video side identification information or teacher classroom teaching behavior multi-view video side identification information. S4 includes the following steps: S41. Use audio and video editing software to process the generated teacher classroom teaching video information or... middle Extract and process video image information from the teacher's classroom teaching session, and generate a set of video frame information from the teacher's classroom teaching session. Or a matrix of multi-perspective live video frames from teachers' classroom teaching The include ;in Indicates the first Video frame information of classroom teaching from various teaching perspectives; include ;in Indicates the first The set of multi-perspective live video frames of teacher classroom teaching corresponding to the teaching perspective type, wherein... include ,in Indicates the The Middle A series of video frames showing teachers' classroom teaching from multiple perspectives; in This represents a collection of live video information from multiple perspectives of classroom teaching by teachers; include , Indicates the number of collections Multi-perspective live video information of teachers' classroom teaching from various teaching perspectives; S42. Establish a video frame set matrix of classroom teaching behaviors of different types of teachers. The include ;in Indicates the first The video frame set of different types of teacher classroom teaching behaviors corresponding to the various types of teacher classroom teaching behaviors; include ,in Indicates the The Middle Video frames showing different types of teachers' classroom teaching behaviors; S43, Based on the Raccoon Optimization Algorithm, the aforementioned The above or the aforementioned The internal description The above With the The internal description The above Perform video image feature matching to search for images that match the provided image. or the aforementioned The matching The corresponding textual information on teacher classroom teaching behaviors is used to generate video-based identification information or multi-view video-based identification information sets of teacher classroom teaching behaviors after data identification. The process generates video-side recognition information of the teacher's classroom teaching behavior or a set of multi-view video-side recognition information of the teacher's classroom teaching behavior. The include ;in Indicates the first Multi-perspective video-based identification information of teachers' classroom teaching behaviors from various teaching perspectives; S5. Perform anomaly detection processing on the audio and video side of teacher classroom teaching behavior, and generate anomaly detection information on the audio and video side of teacher classroom teaching behavior. If there is inconsistency, repeat S1 to S5 until the detection results are consistent. S6. When the results are consistent or when the audio information is not available, perform clustering processing on the teacher classroom teaching behavior recognition results to generate summary information on teacher classroom teaching behavior or summary information on teacher classroom teaching behavior from multiple perspectives. S7. Construct a real-time monitoring system for teachers' classroom teaching behavior and push the results of the real-time monitoring of teachers' classroom teaching behavior to assignments.

2. The intelligent analysis and feedback method for teacher classroom teaching behavior according to claim 1, characterized in that: S1 includes the following steps: S11. Collect audio-visual information of the teacher giving a lecture on the teacher's podium through a camera online, and generate audio-visual information of the teacher's classroom teaching. S12. Using audio and video editing software, the video and audio data in the teacher's classroom teaching scene are decomposed and separated, and the teacher's classroom teaching scene audio information and teacher's classroom teaching scene video information are generated.

3. The intelligent analysis and feedback method for teacher classroom teaching behavior according to claim 2, characterized in that: S2 includes the following steps: S21. Use a breadth-first search algorithm to perform audio data search processing on the audio information of the teacher's classroom teaching, and generate a judgment information on the existence of the teacher's classroom teaching audio based on the audio data search processing results. When audio data is found, the system outputs a message indicating that the audio of the teacher's classroom teaching session exists. If no audio data is found, the output will indicate that the audio of the teacher's classroom teaching does not exist; video information of the teacher lecturing at the podium will be collected from different spatial perspectives using cloud cameras to generate a multi-perspective video information set of the teacher's classroom teaching. And in the multi-perspective live video information collection of teachers' classroom teaching After the data collection is completed, proceed directly to step S4; wherein... include ; Indicates the number of collections Multi-perspective live video information of teachers' classroom teaching from various teaching perspectives.

4. The intelligent analysis and feedback method for teacher classroom teaching behavior according to claim 3, characterized in that: S3 includes the following steps: S31. When the teacher's classroom teaching audio is deemed to exist, the teacher's classroom teaching audio is translated into text using voice translation software, and the teacher's classroom teaching audio text is generated. S32. Establish audio-text information sets of classroom teaching behaviors of different types of teachers. The include ;in Indicates the first Audio text information of different types of teacher classroom teaching behaviors corresponding to various teacher classroom teaching behaviors; S33. Using the KMP search algorithm, the audio and text information of the teacher's classroom teaching is compared with the... The above Perform audio-text character matching to search for characters that match the audio-text information of the teacher's classroom teaching. The corresponding text information of teachers' classroom teaching behavior types is used to generate audio recognition information of teachers' classroom teaching behavior through data identification.

5. The intelligent analysis and feedback method for teacher classroom teaching behavior according to claim 4, characterized in that: S5 includes the following steps: S51. Obtain the audio-side recognition information of the teacher's classroom teaching behavior and the video frame information of the teacher's classroom teaching scene; S52. Match the audio-side recognition information of the teacher's classroom teaching behavior with the video frame information of the teacher's classroom teaching scene using keywords of teacher's classroom teaching behavior type, and generate abnormal detection information of the audio and video side of teacher's classroom teaching behavior based on the keyword matching result of teacher's classroom teaching behavior type; When the keyword matching of the teacher's classroom teaching behavior type is successful, the output of the abnormal detection information of the teacher's classroom teaching behavior audio and video side is consistent; If the keyword for the teacher's classroom teaching behavior type fails to match, the output of the abnormal detection information on the audio and video side of the teacher's classroom teaching behavior is inconsistent. At this time, S1 to S5 are repeated until the abnormal detection information on the audio and video side of the teacher's classroom teaching behavior is consistent.

6. The intelligent analysis and feedback method for teacher classroom teaching behavior according to claim 5, characterized in that: S6 includes the following steps: S61. When the abnormal detection information of the teacher's classroom teaching behavior audio and video is consistent, or when the judgment information of the teacher's classroom teaching scene audio is absent, a clustering search algorithm is used to identify the abnormal information of the teacher's classroom teaching behavior video or the abnormal information of the teacher's classroom teaching behavior audio and video. The above A keyword clustering search was conducted to identify the most frequently occurring keywords related to teachers' classroom teaching behaviors. This search was then used to construct a summary of teachers' classroom teaching behaviors or a summary of their classroom teaching behaviors from multiple perspectives.

7. The intelligent analysis and feedback method for teacher classroom teaching behavior according to claim 6, characterized in that: S7 includes the following steps: S71. The teacher classroom teaching scene audio-visual information, the teacher classroom teaching behavior summary information or the teacher classroom teaching multi-view scene video information and the teacher classroom teaching behavior multi-view summary information are used to construct real-time monitoring information of teacher classroom teaching behavior by data identification. S72. The real-time monitoring information of the teacher's classroom teaching behavior is pushed online to the teaching supervision platform through the Internet of Things communication network, and the real-time monitoring information of the teacher's classroom teaching behavior is displayed and output on the display screen to execute the task of pushing the real-time monitoring results of the teacher's classroom teaching behavior.

8. A teacher classroom teaching behavior intelligent analysis and feedback system, used to implement the teacher classroom teaching behavior intelligent analysis and feedback method according to any one of claims 1-7, characterized in that: The system includes a teacher classroom teaching scene information processing module, a teacher classroom teaching behavior recognition module, and a teacher classroom teaching behavior feedback module.

Images