A method, system and storage medium for identifying invalid scenes in classroom video recording and broadcasting

Abstract

The present invention provides a method, system and storage medium for identifying invalid scenes in classroom recorded videos, including: preprocessing classroom recorded videos, extracting spatial features and human behavior features in images using a convolutional neural network, and outputting labeled frame images; grouping and marking labeled frame images into a sequence by seconds, and using a sequence model to identify the behavioral dynamics of characters, and outputting frame images with multiple time feature labels of characters; using an optical flow method to detect pixel motion between frames; marking invalid scenes in classroom recorded videos according to the frame image labels and pixel motion output by the convolutional neural network and the sequence model; editing and removing invalid scene segments in the recorded videos to generate optimized classroom recorded videos. The present invention significantly improves the recognition accuracy of invalid scenes, avoids misjudgment of valid segments; automatically marks and removes invalid scenes, greatly reduces the storage space requirements of recorded videos, and reduces the workload of manual editing in the later stage.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and computer vision, and in particular to a method, system and storage medium for identifying invalid scenes in classroom recorded and broadcasted videos. Background Art

[0002] At present, classroom recording and broadcasting systems are used to record the teaching process for students to review after class or learn remotely. However, in the actual recording process, invalid scenes often occur, such as no one on the podium, the teacher being still for a long time, or invalid images caused by equipment errors. These invalid scenes increase storage costs and affect video quality and user experience. Existing recording and broadcasting systems lack the function of intelligently identifying invalid scenes, and most of them rely on manual editing, which is inefficient.

[0003] Although there are some anomaly detection technologies for surveillance videos in the prior art, due to the special scenes of classroom recorded videos, the single surveillance video analysis method has low recognition accuracy in this scene. Therefore, there is an urgent need for a method that can effectively identify invalid scenes in classroom recorded videos to improve the effectiveness and quality of recorded videos. Summary of the invention

[0004] In view of the problems existing in the prior art, a method, system and storage medium for identifying invalid scenes in classroom recorded videos are provided, which can automatically identify and eliminate invalid scene fragments in recorded videos.

[0005] The first aspect of the present invention provides a method for identifying invalid scenes in classroom video recording and broadcasting, comprising:

[0006] Preprocess the classroom video recording, and use convolutional neural network to extract the spatial features of the classroom scene and the behavioral features of people in each frame image, and output frame images with corresponding labels;

[0007] The frame images with corresponding labels are grouped into a sequence by second, and the sequence model is used to perform time series analysis to identify the behavioral dynamics of teachers and students, and output frame images with multiple time feature labels of the characters;

[0008] Use optical flow method to detect pixel motion between frames;

[0009] Mark invalid scenes in classroom video recordings based on the frame image labels and pixel motion output by the convolutional neural network and sequence model;

[0010] Edit and remove invalid scenes in the recorded video to generate optimized classroom recorded video.

[0011] As a preferred solution, the pretreatment specifically includes:

[0012] Convert the teacher's recorded video into a frame sequence format;

[0013] Convert each frame of image to HSV color space, and determine the image with area threshold greater than the preset value as abnormal image and remove it;

[0014] Gaussian filtering is used to remove noise from image frames;

[0015] Convert color images to grayscale;

[0016] The processed image frames are named according to preset categories and standardized.

[0017] As a preferred solution, the convolutional neural network includes four two-dimensional convolutional layers and one maximum pooling layer, and is trained using batch normalization to extract spatial features and human behavioral features in classroom scenes.

[0018] As a preferred solution, the sequence model includes two layers of long short-term memory networks, a first convolutional layer, a maximum pooling layer, a second convolutional layer, a global average pooling layer, a batch normalization layer and an output layer connected in sequence, which are used to identify the dynamic behavioral characteristics of teachers and students.

[0019] As a preferred solution, the method of detecting pixel motion between frames using the optical flow method specifically includes:

[0020] Determine the frame images that contain both the character behavior labels output by the convolutional neural network and the teacher-student non-interaction labels output by the sequence model, and take the previous and next frame images to form two sets of data pairs with the current frame respectively;

[0021] The inter-frame pixel motion of two sets of data pairs is detected by the optical flow method, and the smaller value is taken as the motion detection result of the current frame to determine whether there is no significant motion. As a preferred solution, it also includes using the face and gesture detection results to determine whether the teacher is in the podium area and in an effective teaching state, to assist in verifying the feature extraction results of the convolutional neural network and the sequence model.

[0022] As a preferred solution, the method of using the face and gesture detection results to determine whether the teacher is in the podium area and in an effective teaching state specifically includes:

[0023] Use deep learning algorithms to extract facial features of people in frame images, and use classifiers to determine whether they are teachers;

[0024] Then, by detecting the teacher's hand movements, it can be determined whether effective teaching activities are being carried out.

[0025] As a preferred solution, the invalid scenes in the classroom video recording are marked according to the frame image labels and pixel motion conditions output by the convolutional neural network and the sequence model, specifically including:

[0026] Based on the frame image labels and pixel motion output by the convolutional neural network and sequence model, determine whether the frame image meets the following conditions:

[0027] (1) The teacher is not at the podium or remains still for a long time;

[0028] (2) No interaction among students;

[0029] (3) The optical flow method detects no significant motion in the scene;

[0030] If a frame image meets any condition, the frame image is marked as an invalid scene.

[0031] A second aspect of the present invention proposes a system including a memory and a processor, wherein the memory stores a computer program corresponding to the invalid scene identification method for classroom recorded video as described in the first aspect and capable of being loaded and executed by the processor.

[0032] The third aspect of the present invention proposes a computer-readable storage medium having computer program instructions stored thereon. When the program instructions are executed by a processor, they are used to implement the process corresponding to the invalid scene identification method for classroom recorded video described in the first aspect.

[0033] Compared with the prior art, the beneficial effects of adopting the above technical solution are:

[0034] (1) Accurately identify invalid scenes: Based on the combination of image features, time series and behavioral analysis, the recognition accuracy of invalid scenes is significantly improved, avoiding misjudgment of valid clips.

[0035] (2) Reduce storage and manual intervention: Automatically mark and remove invalid scenes, significantly reduce the storage space requirements for recorded videos, and reduce the workload of post-production manual editing.

[0036] (3) Improve viewing experience: By automatically optimizing video content and removing redundant clips, viewers can focus on effective teaching content and improve learning outcomes. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of the invalid scene identification method for classroom recorded video proposed by the present invention. DETAILED DESCRIPTION

[0038] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar modules or modules with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be construed as limitations on the present application. On the contrary, the embodiments of the present application include all changes, modifications and equivalents that fall within the spirit and connotation of the appended claims.

[0039] The current classroom recording and broadcasting system has the following outstanding problems:

[0040] (1) Invalid recording when no one is at the podium: When the teacher is not at the podium or within the camera coverage, the recording system will continue to record invalid content, resulting in empty scenes in the video.

[0041] (2) Teachers or students remain still for a long time: When teachers are playing slides or students are quietly listening to the class, the recorded video lacks actual dynamic content, and the static images cannot provide meaningful learning information for the viewers.

[0042] (3) Invalid scenes caused by equipment failure or error: For example, the camera may mistakenly capture an irrelevant scene or a corner of the classroom, resulting in invalid images being stored in the video file, increasing storage overhead and post-processing work.

[0043] (4) The existence of static redundant clips: Although some clips contain teachers or students, due to the lack of substantive teaching content (such as no teacher-student interaction, only slide presentations), the clips are less helpful for viewers’ learning.

[0044] It can be seen that the recorded videos generated by the classroom recording system often contain many invalid scenes, which not only increases the video storage and processing costs, but also affects the user's viewing experience. Based on this, the present invention proposes a method for identifying invalid scenes in classroom recorded videos. Through the combination of multiple technologies such as convolutional neural networks, LSTM, and optical flow methods, a multi-level scene analysis framework is formed, which can comprehensively evaluate the dynamic and static information in the video and can accurately identify invalid scene fragments in the recorded video. Please refer to Figure 1 , the specific plan is as follows:

[0045] Step 1: Preprocess the classroom recording video.

[0046] In this embodiment, preprocessing is mainly used to remove invalid data of classroom recorded videos, as well as noise reduction and grayscale processing, which helps to improve the accuracy of subsequent image analysis and reduce complexity.

[0047] Specifically, before preprocessing, the classroom recorded video is first converted into a frame sequence form. Multiple video frames can be extracted per second of the video, and the number can be adjusted according to the frame rate of the recording device.

[0048] In this embodiment, each frame of image is converted into HSV color space through OpenCV, and image frames with an area threshold greater than a preset value are determined as abnormal images.

[0049] In order to improve the accuracy of subsequent image analysis, in this embodiment, Gaussian filtering is used to remove noise in the video frame to ensure the clarity of the image. The specific formula is as follows:

[0050]

[0051] Where 𝜎 is the standard deviation of the filter, and x and y are the image coordinates. This processing helps improve the accuracy of subsequent image analysis.

[0052] In order to reduce the computational complexity, in this embodiment, the color image is converted into a grayscale image for subsequent processing. The specific formula is as follows:

[0053]

[0054] Among them, Gray is the grayscale value, R, G, and B represent the values ​​of the red channel (Red), green channel (Green), and blue channel (Blue) of the pixel in the color image respectively. Among them, the weights 0.2989, 0.5870, and 0.1140 reflect the sensitivity of the human eye to the three colors of red, green, and blue respectively.

[0055] For the application scenario of this embodiment, the frame image generated after noise reduction and other processing is named "classroom_time_sequence number, for example, "Jiang'an No. 1 Teaching Building A 101_202011211533_00002.jpg", and placed in the corresponding date folder. Step 2, use the convolutional neural network to extract the spatial features of the classroom scene and the behavioral features of people in each frame image, and output the frame image with corresponding labels.

[0056] In this step, a custom convolutional neural network (CNN) model is used to extract features from the frame image generated in step 1. Specifically, the CNN model used in this embodiment is a CNN model that uses the Keras framework to build four Conv2D (two-dimensional convolution) layers and a MaxPooling (maximum pooling) layer. At the same time, a BatchNormalization (batch normalization) layer is also used to improve training performance.

[0057] Based on the above-mentioned customized convolutional neural network model, the spatial features and human behavior features in the classroom scene can be extracted, and the label and label position data of the frame image and the confidence score can be output. The label can include the object features in the scene or the behavior features of a single person, including but not limited to 0: 'zhuantou', 1: 'zhidao', 2: 'zhanli', 3: 'taolun', 4: 'taitou', 5: 'phone', 6: 'pazhuozi_shuijiao', 7: 'kanshu', 8: 'jiangshou', 9: 'hudong', 10: 'ditou', 11: 'banshu'. Compared with the traditional recognition method based on template matching or simple frame difference, the CNN model proposed in this embodiment can adapt to a variety of different teaching scenarios, has higher adaptability and accuracy, and improves the recognition ability of complex classroom scenes without relying on fixed templates.

[0058] Step 3: Group the frame images with corresponding labels generated in step 2 into a sequence by seconds, and use the sequence model to perform time series analysis to identify the behavioral dynamics of teachers and students, and output frame images with multiple time feature labels of the characters.

[0059] In this embodiment, the long short-term memory network (LSTM) is used to perform time series analysis on video frames to identify the behavioral changes of teachers and students. By analyzing the time dependency of actions, it is possible to accurately determine whether the lecturer is conducting effective teaching activities and distinguish invalid scenes with long periods of stillness.

[0060] Specifically, by grouping the labeled frame images generated in step 2 into a sequence by second and inputting it into the sequence model for analysis, the behavior dynamics of the teacher and the student can be realized. The sequence model includes:

[0061] (1) Two-layer LSTM network, each layer has 32 neurons, and the activation function used is ReLU.

[0062] (2) The first convolutional layer contains 128 neurons.

[0063] (3) Maximum pooling layer, used to perform downsampling operations.

[0064] (4) The second pooling layer, the second convolutional layer contains 256 neurons.

[0065] (5) Global average pooling layer, which converts multi-D feature maps into 1-D feature vectors.

[0066] (6) Batch normalization layer, used to assist model convergence.

[0067] (7) Output layer, which is a fully connected layer with multiple neurons of the SoftMax classifier layer. Each neuron of the (1D vector after feature extraction) is connected to each neuron of the current layer to generate a global feature representation. The dimension of the input feature vector is the output dimension of the (global average pooling layer) (e.g., 256 values), and the output dimension is equal to the predefined number of categories N. The output is then mapped to a probability distribution through SoftMax, ranging from [0, 1], and the sum of all output probabilities is 1. This layer represents the probability of the set label, for example, the labels can include: long-term slide show, no interaction between teachers and students, camera switching, etc.

[0068] In this embodiment, the LSTM model is used to obtain multiple time feature labels and probability data of frame images and characters in the frame images, which can capture complex time dependencies and accurately identify the behavior dynamics of teachers and students. Even if the behavior changes slowly, it can be correctly identified as a valid scene. It is especially suitable for long-term classroom recording and broadcast video analysis, which improves the recognition accuracy of invalid scenes.

[0069] Step 4: Use the optical flow method to detect pixel motion between frames.

[0070] In this embodiment, the optical flow method is mainly used to detect the pixel motion between consecutive frames to determine the dynamic changes in the scene. When there is no significant motion in the scene, it can be automatically determined as an invalid scene, which is of great significance in classroom video recording, because when teachers or students are generally still for a long time, the video does not contain meaningful content.

[0071] In this embodiment, optical flow detection is only performed on frame images that contain both character behavior labels (generated by step 2) and teacher-student non-interaction labels (generated by step 3). Since classroom scenes generally use cameras to record videos and only use single-frame calculations, there are accuracy issues. Based on this, in this embodiment, one frame of each current frame is taken together with the current frame to form two sets of data pairs. For these two sets of data pairs, the optical flow method is used to analyze the movement of pixels between frames, and the smaller result is taken as the result of the current frame, and it is determined whether it is marked as a frame image without significant movement. In this embodiment, the optical flow method uses the Lucas-Kanade algorithm, and the Sobel operator is used to calculate the spatial gradient and temporal gradient. The optical flow equation is constructed based on the composed data pairs, as follows:

[0072]

[0073] Among them, I x ,I y Represent the spatial gradient of the image in the x and y directions, V x and V yis the speed of the pixel in these two directions, I t is the temporal gradient. Finally, the optical flow vector is solved by the least squares method, the motion size and direction of each pixel are calculated, and the average optical flow amplitude of the entire frame image is obtained. A threshold is set according to the average optical flow amplitude. If the average optical flow amplitude is less than the set threshold, the frame is judged as a frame image without significant motion and marked.

[0074] The optical flow method can more flexibly detect scenes with no movement for a long time, especially for scenes with subtle movements (such as occasional movements of teachers), and can make more accurate judgments.

[0075] Step 5: Mark invalid scenes in classroom video recordings based on the behavioral dynamics and pixel movement of teachers and students.

[0076] Combined with the detection results of step 2 to step 4, invalid scenes are judged by multiple conditions and marked. Specifically, as long as a frame image meets any of the following conditions, the frame image is marked as an invalid scene:

[0077] (1) The teacher is not at the podium or remains still for a long time;

[0078] (2) No interaction among students;

[0079] (3) The optical flow method detects no significant motion in the scene.

[0080] Step 6: Edit and remove invalid scenes from the recorded video to generate an optimized classroom recorded video.

[0081] When the invalid scene recognition method proposed in the present invention is applied to the recording and broadcasting system, the system can automatically edit the marked invalid scenes, remove the invalid segments, and generate optimized recorded and broadcasted videos. The editing process is based on time tags, and the invalid segments are cut out and reassembled into a complete video using known video editing algorithms (such as FFmpeg). The automatic marking and editing functions reduce the workload of manual processing and improve the quality and storage efficiency of the video.

[0082] In order to improve the recognition accuracy of invalid scenes, in one embodiment, the existing computer vision (CV) is used to determine whether the teacher is in the podium area, and the existing gesture detection is combined to determine whether the teacher is conducting effective teaching activities. This method improves the recognition ability of the lecturer's activities and can distinguish whether the lecturer is only in the podium position but not actually teaching. Specifically, for face detection, the frame image of step 4 is input in this embodiment, and a deep learning algorithm (such as ResNet) is used to extract facial features, and a Softmax classifier is used to determine whether it is a teacher. The formula is as follows:

[0083]

[0084] Among them, X is the facial feature vector, W is the weight matrix, y is the classification result, and b is the bias term used to adjust the model output.

[0085] After detecting the teacher using the face, by detecting the teacher's hand movements, it is possible to determine whether effective teaching activities (such as writing, demonstrating, etc.) are being carried out.

[0086] This process can be carried out simultaneously with the feature extraction process of the convolutional neural network and sequence model in steps 2 and 3. By comparing the confidence of the extraction results and correcting the extraction results according to actual needs, the recognition accuracy can be effectively improved.

[0087] The present invention forms a multi-level scene analysis framework through the combination of convolutional neural networks, LSTM, optical flow method and other technologies, which can comprehensively evaluate the dynamic and static information in the video and realize the automatic detection, marking and processing of invalid scenes.

[0088] An embodiment of the present invention further proposes a system, including a memory and a processor, wherein the memory stores a computer program corresponding to the aforementioned invalid scene identification method for classroom recorded videos that can be loaded and executed by the processor.

[0089] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program includes a program code for executing the method shown in the flowchart.

[0090] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0091] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0092] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0093] As another aspect, the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the invalid scene recognition method for classroom recorded video described in the above embodiment.

[0094] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the invalid scene recognition method for classroom recorded video described in the above embodiment.

[0095] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0096] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation method of the present application.

[0097] For those skilled in the art, the specific meanings of the above terms in the present invention can be understood in specific situations; the drawings in the embodiments are used to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0098] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for identifying invalid scenes in classroom video recording and broadcasting, characterized in that: include: Preprocess the classroom video recording, and use convolutional neural network to extract the spatial features of the classroom scene and the behavioral features of people in each frame image, and output frame images with corresponding labels; The frame images with corresponding labels are grouped into a sequence by second, and the sequence model is used to perform time series analysis to identify the behavioral dynamics of teachers and students, and output frame images with multiple time feature labels of the characters; Use optical flow method to detect pixel motion between frames; Mark invalid scenes in classroom video recordings based on the frame image labels and pixel motion output by the convolutional neural network and sequence model; Edit and remove invalid scenes from the recorded video to generate optimized classroom recorded video; It also includes using the face and gesture detection results to determine whether the teacher is in the podium area and in an effective teaching state, and assisting in verifying the feature extraction results of the convolutional neural network and the sequence model. Specifically, it includes: using a deep learning algorithm to extract facial features of people in the frame image, and using a classifier to determine whether it is a teacher; then by detecting the teacher's hand movements, it is determined whether effective teaching activities are being carried out; The invalid scenes in the classroom video recording are marked according to the frame image labels and pixel motion conditions output by the convolutional neural network and the sequence model, specifically including: Based on the frame image labels and pixel motion output by the convolutional neural network and sequence model, determine whether the frame image meets the following conditions: (1) The teacher is not at the podium or remains still for a long time; (2) No interaction among students; (3) The optical flow method detects no significant motion in the scene; If a frame image meets any condition, the frame image is marked as an invalid scene.

2. The method for identifying invalid scenes in classroom video recording and broadcasting according to claim 1, characterized in that: The pre-processing specifically includes: Convert the teacher's recorded video into a frame sequence format; Convert each frame of image to HSV color space, and identify images with area threshold greater than the preset value as abnormal images and remove them; Gaussian filtering is used to remove noise from image frames; Convert color images to grayscale; The processed image frames are named according to preset categories and standardized.

3. The method for identifying invalid scenes in classroom video recording and broadcasting according to claim 1, characterized in that: The convolutional neural network includes four two-dimensional convolutional layers and one maximum pooling layer, and is trained using batch normalization to extract spatial features and human behavioral features in classroom scenes.

4. The method for identifying invalid scenes in classroom video recording and broadcasting according to claim 1, characterized in that: The sequence model includes two layers of long short-term memory networks, a first convolutional layer, a maximum pooling layer, a second convolutional layer, a global average pooling layer, a batch normalization layer and an output layer connected in sequence, and is used to identify the dynamic characteristics of the behavior of teachers and students.

5. The method for identifying invalid scenes in classroom video recording and broadcasting according to claim 1, characterized in that: The method of using the optical flow method to detect pixel motion between frames specifically includes: Determine the frame images that contain both the character behavior labels output by the convolutional neural network and the teacher-student non-interaction labels output by the sequence model, and take the previous and next frame images to form two sets of data pairs with the current frame respectively; The optical flow method is used to detect the inter-frame pixel motion of two sets of data pairs, and the minimum result is taken as the motion detection result of the current frame to determine whether there is no significant motion.

6. A system, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program corresponding to the invalid scene identification method for classroom recorded video as described in any one of claims 1 to 5, which can be loaded by the processor and executed.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by the processor, they are used to implement the process corresponding to the invalid scene identification method for classroom recorded video described in any one of claims 1 to 5.

Images