Network teaching video knowledge point classification extraction method and device, equipment and medium

Abstract

The invention relates to a network teaching video knowledge point classification extraction method and device, equipment and a medium, and relates to the field of video processing, the method comprises the following steps: using a preset character recognition model to extract text data of each video key frame in a video key frame sequence to generate a text sequence corresponding to each video key frame; inputting a text sequence corresponding to the video key frame into a knowledge point classification model trained to a convergence state, and generating a corresponding knowledge point label for each video key frame; and based on the timestamps corresponding to the video key frames and the knowledge point tags, cutting the network teaching video into a plurality of video clips, and merging the video clips corresponding to the same knowledge point tag to determine the network teaching video corresponding to each knowledge point tag. According to the invention, the network teaching video corresponding to the required learning and teaching content can be quickly and accurately found, and the learning and teaching efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of video processing, and in particular to a method for classifying and extracting knowledge points from online teaching videos, a corresponding device, an electronic device, and a computer-readable storage medium. Background Art

[0002] In modern education, online teaching videos have become an important teaching tool. The video teaching content is rich and intuitive, which helps to improve learners' interest and comprehension. However, existing video teaching resources usually lack systematic organization and efficient classification methods. Teaching videos of different subjects, different stages and different knowledge points are often scattered. Especially for students with specific learning needs, it is difficult to quickly locate the required knowledge points, which seriously affects the utilization efficiency of educational video resources.

[0003] In traditional technologies, video classification and labeling mostly rely on manual operations, and their efficiency and accuracy need to be improved. However, with the continuous increase in video content, traditional methods are becoming increasingly unable to meet the needs.

[0004] When processing input videos, existing technologies recognize each image frame by frame, which requires a huge amount of calculation. In addition, when using convolutional neural networks to extract video features, due to the large number of layers and the large number of convolution kernels in each layer of these models, the amount of calculation and memory requirements are very high, which has great limitations in real-time and efficiency. At the same time, additional processing of educational courseware is required; in the OCR text detection process, Faster R-CNN and FCN are not effective when dealing with complex backgrounds and small-sized texts, and cannot accurately identify detailed information. TextBoxes and CTPN do not provide sufficient support for curved text, and the detection speed is slow, which affects real-time performance.

[0005] To sum up, the processing of videos in the existing technology has very high computing and memory requirements, which leads to great limitations in real-time and efficiency, as well as poor results when processing complex backgrounds and small-size texts, and inability to accurately identify detailed information. The applicant has made corresponding explorations in order to solve this problem. Summary of the invention

[0006] The purpose of this application is to solve the above-mentioned problems and to provide a method for classifying and extracting knowledge points from online teaching videos, a corresponding device, an electronic device and a computer-readable storage medium.

[0007] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0008] A method for classifying and extracting knowledge points from online teaching videos proposed to meet one of the purposes of this application includes:

[0009] In response to an instruction to classify and extract knowledge points from online teaching videos, a collection of online teaching videos is obtained, wherein the collection of online teaching videos is constructed from a plurality of online teaching videos of different types;

[0010] Using a video key frame recognition model that has been trained to a convergence state, extracting each video key frame from a video stream corresponding to the online teaching video to construct a video key frame sequence;

[0011] Extracting text data of each video key frame in the video key frame sequence using a preset character recognition model to generate a text sequence corresponding to each video key frame;

[0012] Inputting the text sequence corresponding to the video key frame into the knowledge point classification model that has been trained to a convergence state, generating a corresponding knowledge point label for each video key frame, wherein the knowledge point label represents the subject and its corresponding knowledge point category;

[0013] Based on the timestamps and knowledge point labels corresponding to the video key frames, the online teaching video is cut into multiple video clips, and the video clips corresponding to the same knowledge point label are merged to determine the online teaching video corresponding to each knowledge point label, so as to complete the classification extraction of knowledge points in the online teaching video.

[0014] Optionally, before the step of using a video key frame recognition model that has been trained to a convergent state to extract each video key frame from a video stream corresponding to the online teaching video to construct a video key frame sequence, the step includes:

[0015] In response to the preprocessing instruction, an original frame rate of a video stream corresponding to the online teaching video is obtained, wherein the original frame rate represents the number of image frames per second;

[0016] Based on the original frame rate of the video stream corresponding to the online teaching video, video image frames in the video stream corresponding to the online teaching video are extracted at preset time intervals as target video image frames of the online teaching video.

[0017] Optionally, the step of extracting each video key frame from the video stream corresponding to the online teaching video using a video key frame recognition model that has been trained to a convergence state to construct a video key frame sequence includes:

[0018] Determine a target video image frame of the online teaching video, and input the target video image frame into a video key frame recognition model that has been trained to a convergence state;

[0019] Perform image block embedding on the input target video image frame to obtain multiple image block vectors, and add a classification vector to form multiple embedding vectors;

[0020] Adding a position encoding vector to the multiple embedding vectors to form an input vector, wherein the position encoding vector is used to maintain spatial position information between image blocks;

[0021] Stacking a plurality of encoding modules for the input vector to perform feature extraction to determine a feature vector corresponding to each target video image frame, wherein the encoding module includes a plurality of multi-head self-attention encoding layers;

[0022] The Euclidean distance between the feature vectors of adjacent video image frames in the target video image frame is calculated and determined, and whether the Euclidean distance exceeds a preset change rate threshold is detected. If exceeded, the target video image frame exceeding the preset change rate threshold is used as a video key frame to construct a video key frame sequence.

[0023] Optionally, the step of extracting text data of each video key frame in the key video image sequence by using a preset character recognition model to generate a text sequence corresponding to each video key frame includes:

[0024] Get the video key frame containing the characters to be recognized;

[0025] Input the video key frame containing the characters to be recognized into a preset character recognition model, wherein the character recognition model is constructed by a DB text detection algorithm module, a text direction classifier and a CRNN text recognition algorithm module, and a text direction classifier is provided between the output of the DB text detection algorithm module and the input of the CRNN text recognition algorithm module;

[0026] The key frame sequence number and text data corresponding to the video key frame are output to generate a text sequence corresponding to each video key frame.

[0027] Optionally, the steps of training the knowledge point classification model include:

[0028] Obtaining a text sequence corresponding to any video key frame of an online teaching video in an online education platform as a training sample;

[0029] Constructing a coding vector from the training sample, calling a text feature extractor in a preset knowledge point classification model to extract text features and obtain corresponding text features;

[0030] A multi-layer fully connected neural network is called to classify each text feature to obtain the classification result. The subject pointed to by the training sample and its corresponding knowledge point category are used as the supervision label of the classification result. The weight parameters of the text feature extractor are corrected by backpropagation until its loss function converges to complete the training.

[0031] Optionally, based on the timestamps and knowledge point labels corresponding to the video key frames, the online teaching video is cut into multiple video clips, and the video clips corresponding to the same knowledge point label are merged to determine the steps of the online teaching video corresponding to each knowledge point label, including:

[0032] Obtaining video key frames in the online teaching video, and recording the timestamp of each video key frame in the online teaching video, wherein the timestamp is used to identify the precise position of each key frame;

[0033] Based on the timestamps and knowledge point labels corresponding to the video key frames, the positions of the video key frames in the online teaching video are read to determine the video segmentation points;

[0034] Segmenting the online teaching video based on the video segmentation points to determine video segments corresponding to each knowledge point label, wherein the video segments correspond to the knowledge point labels one by one;

[0035] The video clips corresponding to the same knowledge point label are merged to determine the online teaching videos corresponding to each knowledge point label, so as to complete the classification extraction of knowledge points in the online teaching videos.

[0036] Optionally, the basic network architecture of the video key frame recognition model is a Vi T model; and the basic network architecture of the knowledge point classification model is a BERT model.

[0037] A network teaching video knowledge point classification and extraction device provided for another purpose of the present application includes:

[0038] A teaching video acquisition module, configured to respond to an instruction to classify and extract knowledge points from online teaching videos, and acquire an online teaching video set, wherein the online teaching video set is constructed from a plurality of online teaching videos of different types;

[0039] A key frame sequence construction module is configured to extract each video key frame from the video stream corresponding to the online teaching video using a video key frame recognition model that has been trained to a convergence state to construct a video key frame sequence;

[0040] A text sequence generation module is configured to extract text data of each video key frame in the video key frame sequence using a preset character recognition model to generate a text sequence corresponding to each video key frame;

[0041] A knowledge point label generation module is configured to input the text sequence corresponding to the video key frame into the knowledge point classification model that has been trained to a convergence state, and generate a corresponding knowledge point label for each video key frame, wherein the knowledge point label represents a subject and its corresponding knowledge point category;

[0042] The classification extraction module is configured to cut the online teaching video into multiple video clips based on the timestamps and knowledge point labels corresponding to the video key frames, merge the video clips corresponding to the same knowledge point labels to determine the online teaching videos corresponding to the various knowledge point labels, so as to complete the classification extraction of knowledge points in the online teaching video.

[0043] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the method for classifying and extracting knowledge points from online teaching videos described in the present application.

[0044] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the method for classifying and extracting knowledge points from online teaching videos in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0045] Compared with the prior art, the present application aims at the problems that the processing of videos in the prior art has very high computational complexity and memory requirements, which results in great limitations in real-time and efficiency, as well as poor effects when processing complex backgrounds and small-size texts, and inability to accurately identify detailed information. The present application includes but is not limited to the following beneficial effects:

[0046] First, the network teaching video knowledge point classification and extraction method proposed in this application is aimed at traditional teaching videos that require manual content annotation and classification, which is not only time-consuming and labor-intensive, but also prone to human errors and other problems; this application automatically extracts features and classifies knowledge points through video key frame recognition models, character recognition models, and knowledge point classification models, and can quickly and accurately annotate video content to improve the utilization efficiency of teaching resources; students and teachers can quickly find the network teaching videos corresponding to the required learning and teaching content through the classified knowledge points, saving time, improving the efficiency of learning and teaching, effectively improving the flexibility of online learning, and improving user efficiency;

[0047] Secondly, the video key frame recognition model proposed in this application adopts the deep learning model of attention mechanism for training, which greatly reduces the number of videos converted into images, relieves the calculation pressure of subsequent modules, and improves the overall operation efficiency; at the same time, the generated key frame images have low similarity and strong representativeness, which can be used as an overview of the key content of the video; through this method, ordinary computers can make full use of their computing power to run more advanced deep learning algorithms, significantly improving the performance of the video frame feature extraction algorithm;

[0048] Third, the character recognition model proposed in this application is constructed by a DB text detection algorithm module, a text direction classifier and a CRNN text recognition algorithm module. A text direction classifier is provided between the output of the DB text detection algorithm module and the input of the CRNN text recognition algorithm module to cope with text recognition in different directions and improve the accuracy of text recognition. At the same time, the CRNN text recognition algorithm is suitable for processing complex texts, and is suitable for complex and irregular texts, such as handwriting in lectures or teaching videos or texts with different fonts. It has strong robustness and is suitable for processing video key frames in course videos recorded during the teaching, academic lectures, academic conferences and scientific research training of this application, and classifying and extracting knowledge points of specific contents of the video key frames according to the recognition results;

[0049] Fourthly, this application realizes the accurate segmentation of different knowledge points extracted from long videos. This method ensures that the final generated video file has good playback continuity through optimized video merging technology, meeting the needs of actual teaching scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0051] Figure 1 A schematic diagram of the process of classifying and extracting knowledge points from online teaching videos in an embodiment of the present application;

[0052] Figure 2 This is a schematic diagram of the process of preprocessing the online teaching video in the embodiment of the present application;

[0053] Figure 3 A schematic diagram of a process for constructing a video key frame sequence in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of a process for generating a text sequence corresponding to each video key frame in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the process of training a knowledge point classification model in an embodiment of the present application;

[0056] Figure 6 A schematic diagram of a process for determining the online teaching video corresponding to each knowledge point label in an embodiment of the present application;

[0057] Figure 7 This is a functional block diagram of a device for classifying and extracting knowledge points from online teaching videos in an embodiment of the present application;

[0058] Figure 8 It is a schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0059] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0060] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0061] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0062] It will be understood by those skilled in the art that the "client", "terminal" and "terminal device" used herein include both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service, personal communication system), which can combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant, personal digital assistant), which may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar and / or GPS (Global Positioning System, global positioning system) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "client", "terminal" and "terminal device" used herein may be portable, transportable, installed in a vehicle (air, sea and / or land), or suitable for and / or configured to run locally, and / or in a distributed form, at any other location on the earth and / or in space. The "client", "terminal" and "terminal device" used herein may also be a communication terminal, an Internet terminal, a music / video playing terminal, for example, a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playing function, or a smart TV, a set-top box and other devices.

[0063] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0064] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.

[0065] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for access.

[0066] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0067] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.

[0068] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.

[0069] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0070] See also Figure 1 In one embodiment, the network teaching video knowledge point classification and extraction method of the present application includes:

[0071] Step S10, in response to an instruction to classify and extract knowledge points from online teaching videos, obtaining an online teaching video collection, wherein the online teaching video collection is constructed from a plurality of online teaching videos of different types;

[0072] The online teaching video management terminal can respond to the instruction of classifying and extracting the knowledge points of the online teaching video, and obtain the online teaching video set, wherein the online teaching video set is constructed by multiple different types of online teaching videos; specifically, the online teaching video can be a course video recorded during teaching, academic lectures, academic conferences and scientific research training, etc. When importing a single online teaching video, it supports multiple video formats, including but not limited to online teaching videos in video formats such as MP4, AVI, and WMV. At the same time, it can upload to the database backend, support the upload of large amounts of video, and ensure that the video data is safely and stably stored in the server of the online teaching video management terminal.

[0073] In some embodiments, see Figure 2 , before the step of extracting each video key frame from the video stream corresponding to the online teaching video using the video key frame recognition model that has been trained to a convergence state to construct a video key frame sequence, the step includes:

[0074] Step S101, responding to the preprocessing instruction, obtaining the original frame rate of the video stream corresponding to the online teaching video, wherein the original frame rate represents the number of image frames per second;

[0075] Step S102: based on the original frame rate of the video stream corresponding to the online teaching video, extract the video image frames in the video stream corresponding to the online teaching video at preset time intervals as the target video image frames of the online teaching video.

[0076] Specifically, before extracting each video key frame from the video stream corresponding to the online teaching video using the video key frame recognition model that has been trained to a convergence state, the online teaching video management terminal can respond to an instruction to preprocess the online teaching video to obtain the original frame rate (fps) of the video stream corresponding to the online teaching video, wherein the original frame rate (fps) represents the number of image frames per second; then, according to the original frame rate of the online teaching video, the video image frames are downsampled using a fixed interval method to reduce the number of video image frames, and the video image frames in the video stream corresponding to the online teaching video are extracted at preset time intervals as the target video image frames of the online teaching video. For example, if the target frame rate is 10 frames per second, 10 frames are extracted per second.

[0077] The above steps help to reduce the amount of video data, improve the efficiency of subsequent video key frame processing, and save computing resources.

[0078] Step S20, using a video key frame recognition model that has been trained to a convergence state to extract each video key frame from a video stream corresponding to the online teaching video to construct a video key frame sequence;

[0079] After determining the target video image frame of the online teaching video, a video key frame recognition model that has been trained to a convergence state is used to extract each video key frame from the video stream corresponding to the online teaching video to construct a video key frame sequence; wherein the basic network architecture of the video key frame recognition model is a Vi T model.

[0080] Specifically, see Figure 3 The step of extracting each video key frame from the video stream corresponding to the online teaching video using the video key frame recognition model that has been trained to a convergence state to construct a video key frame sequence includes:

[0081] Step S201, determining a target video image frame of an online teaching video, and inputting the target video image frame into a video key frame recognition model that has been trained to a convergence state;

[0082] After determining the target video image frame of the online teaching video, the target video image frame is input into a video key frame recognition model that has been trained to a convergence state to extract the video key frame in the target video image frame of the online teaching video.

[0083] Step S202: embedding an image block for the input target video image frame to obtain multiple image block vectors, and adding a classification vector to form multiple embedding vectors;

[0084] The basic network architecture of the video key frame recognition model is the ViT model, which is a model that is further improved based on the Transformer model applied to NLP problems and acts on the image field. It first converts images in the image field into word structures in natural language processing, which is called image block embedding.

[0085] Specifically, the input target video image frame is standardized to obtain a picture of standard size, which can be regarded as a complete sentence; then it is divided into small blocks of fixed size, called Patches, and the pixel values ​​of each small block are flattened to become a word in the sentence. Subsequently, each Patch is compressed into a vector of a certain dimension through a fully connected network to obtain multiple image block vectors. This process is called image block embedding (PatchEmbedding). In an embodiment of the present application, the standard size is 224*224, the fixed size of Patch is 16*16, and the dimension of the vector is 768. Those skilled in the art can determine the above parameters as needed according to the actual scenario, and are not limited here.

[0086] After obtaining multiple image block vectors, a classification vector of the same dimension is concatenated. It is not difficult to understand that this vector is used for category information learning in the model training process, and the classification vector is a learnable embedding vector.

[0087] Step S203: adding a position encoding vector to the multiple embedding vectors to form an input vector, wherein the position encoding vector is used to maintain spatial position information between image blocks;

[0088] Multiple embedding vectors can be obtained from the above step S202, but the embedding vectors lack position information, that is, except for the classification vector, other vectors have their corresponding position information in the picture. Therefore, in order to maintain the spatial position information between each patch in the input image, it is necessary to add a position encoding vector to these multiple embedding vectors.

[0089] Specifically, a one-dimensional learnable position embedding vector is directly used, and the position embedding vector is directly added to the embedding vector to form an input vector. Therefore, the input target video image frame has completed the vector embedding work here and can be input into the Transformer module for training and feature learning.

[0090] Step S204: stacking multiple encoding modules for the input vector to perform feature extraction to determine a feature vector corresponding to each target video image frame, wherein the encoding module includes multiple multi-head self-attention encoding layers;

[0091] Multiple encoding modules are stacked for feature extraction on the above input vector to determine the feature vector corresponding to each target video image frame, wherein the encoding module mainly includes multiple multi-head self-attention encoding layers and multi-layer perceptrons; specifically, the encoding module includes two parts, the first part is layer normalization (Layer Norm) -> multi-head self-attention (Mult-Head Attention) -> Dropout -> Short Cut. The second part is layer normalization (Layer Norm) -> multi-layer perceptron (Multi-Layer Perception) -> Dropout -> ShortCut. The multi-layer perceptron includes full connection (Linear) -> activation function (GELU) -> Dropout -> full connection (Linear) -> Dropout; the layer normalization is to normalize the specified dimension of a single data; the feature extraction enables the classification vector to fuse the semantic features of all image blocks to determine the feature vector corresponding to each target video image frame.

[0092] Step S205, calculate and determine the Euclidean distance between the feature vectors of adjacent video image frames in the target video image frame, detect whether the Euclidean distance exceeds a preset change rate threshold, and if so, use the target video image frame that exceeds the preset change rate threshold as a video key frame to construct a video key frame sequence.

[0093] Specifically, after determining the feature vector corresponding to each target video image frame, a two-dimensional array is constructed: the feature vector extracted from each frame is stored in the two-dimensional array features_array, where each row represents the feature vector of a frame, and the feature vector is obtained from the last hidden state of the ViT model; the feature vectors in the two-dimensional array features_array are stored in a list named features for subsequent processing.

[0094] The Euclidean distance between the feature vectors of adjacent video image frames in the target video image frame is calculated to measure the rate of change of the feature. The calculation formula of the Euclidean distance between the feature vectors of adjacent video image frames (the difference between the feature vectors of adjacent video image frames) is expressed as follows:

[0095]

[0096] Among them, a and b are two eigenvectors, and d is the dimension of the eigenvector.

[0097] Furthermore, according to the Euclidean distance between the feature vectors of adjacent video image frames, that is, the difference between the feature vectors of adjacent video image frames, an array with a shape of (n-1, d) is obtained, where n is the number of frames and d is the dimension of the feature vector.

[0098] Furthermore, a video key frame is selected from the target video image frame according to a preset change rate threshold (threshold), wherein the preset change rate threshold (threshold) takes a value between 0.1 and 10. In this embodiment, the preset change rate threshold (threshold) is taken as 2, and the target video image frame whose change rate r is greater than the preset change rate threshold is considered to be a video key frame.

[0099] Furthermore, the image serial numbers frame_id of the detected video key frames are output, and these frames are identified as video key frames in the online teaching video.

[0100] Step S30, using a preset character recognition model to extract text data of each video key frame in the video key frame sequence to generate a text sequence corresponding to each video key frame;

[0101] A video key frame recognition model that has been trained to a convergence state is used to extract each video key frame from the video stream corresponding to the online teaching video to construct a video key frame sequence, and then a preset character recognition model is used to extract text data of each video key frame in the video key frame sequence to generate a text sequence corresponding to each video key frame; wherein the character recognition model is constructed by a DB text detection algorithm module, a text direction classifier and a CRNN text recognition algorithm module, and a text direction classifier is provided between the output of the DB text detection algorithm module and the input of the CRNN text recognition algorithm module.

[0102] For further information, see Figure 4 The step of extracting text data of each video key frame in the key video image sequence using a preset character recognition model to generate a text sequence corresponding to each video key frame comprises:

[0103] Step S301, obtaining a video key frame containing characters to be recognized;

[0104] Step S302: inputting the video key frame containing the characters to be recognized into a preset character recognition model, wherein the character recognition model is constructed by a DB text detection algorithm module, a text direction classifier and a CRNN text recognition algorithm module, and a text direction classifier is provided between the output of the DB text detection algorithm module and the input of the CRNN text recognition algorithm module;

[0105] Step S303: output the key frame sequence number and text data corresponding to the video key frame to generate a text sequence corresponding to each video key frame.

[0106] After constructing the video key frame sequence in the online teaching video, obtain the video key frame containing the characters to be recognized from the video key frame sequence; input the video key frame containing the characters to be recognized into a preset character recognition model, wherein the character recognition model is constructed by a DB text detection algorithm module, a text direction classifier and a CRNN text recognition algorithm module, and a text direction classifier is provided between the output of the DB text detection algorithm module and the input of the CRNN text recognition algorithm module; output the key frame sequence number and text data corresponding to the video key frame to generate a text sequence corresponding to each video key frame.

[0107] Specifically, the DB (Differentiable Binarization) text detection algorithm is used to detect text areas from the video key frames of this application. The core idea is to convert the text detection problem into a binarization task and optimize it through a deep learning method. The processing flow includes:

[0108] First, a convolutional neural network (CNN) is used to extract features from the input image. Commonly used networks include ResNet, VGG, etc., which are all mature neural networks and can be used any one. Among them, the feature map extracted by the network contains the local information of the image and high-level feature representation.

[0109] Furthermore, the feature map is input into a segmentation network, which is responsible for generating a binary text region prediction map. In the image output by the segmentation network, the text region is marked as foreground (usually white) and the background is marked as background (usually black). The differentiated binary loss function enables the network to generate clear text region boundaries.

[0110] Through further contour detection or morphological operations, the connected regions in the binary image are extracted and the bounding box of the text region is located to determine the text image region.

[0111] From the above steps, we can see that the DB text detection algorithm effectively identifies text image areas by converting the text detection task into a binary problem and using a deep learning network for feature extraction and segmentation. Its advantage is that it can handle various complex scenes and text styles. At the same time, it improves detection accuracy and stability through differential loss functions and post-processing techniques.

[0112] In some embodiments, a text direction classifier is provided between the output of the DB text detection algorithm module and the input of the CRNN text recognition algorithm module, and the text direction classifier is used to predict the direction of the text area and perform corresponding rotation correction. The text image area output by the DB text detection algorithm module is used as the input of the text direction classifier, and a convolutional neural network (CNN) is used to extract features, which is expressed as f=CNN(I). The convolutional neural network (CNN) extracts effective features from the text image area for subsequent classification tasks;

[0113] The Softmax layer is used to classify the extracted features and calculate the output D = Softmax (W f + b), where W and b are the trained weights and biases. The classification result D includes four directions: forward (0 degrees), 90 degrees counterclockwise rotation, 180 degrees counterclockwise rotation, and 270 degrees counterclockwise rotation.

[0114] According to the classification result D, the image is rotated and corrected to adjust the text area to the positive direction.

[0115] In some embodiments, the CRNN text recognition algorithm module recognizes the text area after rotation correction. The CRNN (Convolutional Recurrent Neural Network) text recognition algorithm combines the advantages of convolutional neural network (CNN) and recurrent neural network (RNN) to process text recognition tasks in sequence data. Its workflow includes:

[0116] In the convolutional layer, a convolutional neural network is used to extract features in the image. The convolutional layer is used to extract local features and spatial information from the input video key frames;

[0117] In the pooling layer, the dimension of the feature map is reduced, thereby reducing the amount of calculation while retaining important feature information; in the recursive layer, the features extracted by the convolutional layer are input into a recursive neural network (such as LSTM or GRU). The recursive layer is used to process the temporal dependency of sequence data and to understand the contextual relationship between characters; the connectionist temporal classification (CTC) loss function is used for training, allowing the model to learn without explicit alignment in the input sequence, thereby predicting characters and decoding sequences.

[0118] From the above steps, we can see that the CRNN (Convolutional Recurrent Neural Network) text recognition algorithm extracts features through the convolution layer, then processes the sequence characteristics through the recursive layer, and finally performs text recognition through CTC decoding. CRNN combines the advantages of convolutional neural network (CNN) and recurrent neural network (RNN). Convolutional neural network (CNN) is good at extracting local features from images, while recurrent neural network (RNN) is good at processing sequence data. The CRNN text recognition algorithm combines the two and can effectively process and recognize character sequences. Unlike traditional OCR methods, the CRNN text recognition algorithm can process variable-length text sequences without pre-fixing the length of the character sequence. The CRNN text recognition algorithm can be trained directly from image to text end-to-end, which simplifies the training process of the model.

[0119] Furthermore, the CRNN text recognition algorithm is suitable for processing complex texts, and is suitable for complex and irregular texts, such as handwriting or texts with different fonts in lectures or teaching videos. It has strong robustness and is suitable for processing video key frames in course videos recorded during the teaching, academic lectures, academic conferences and scientific research training of this application.

[0120] Step S40: input the text sequence corresponding to the video key frame into the knowledge point classification model that has been trained to a convergence state, and generate a corresponding knowledge point label for each video key frame, wherein the knowledge point label represents the subject and its corresponding knowledge point category;

[0121] A preset character recognition model is used to extract text data of each video key frame in the video key frame sequence to generate a text sequence corresponding to each video key frame, and then the text sequence corresponding to the video key frame is input into a knowledge point classification model that has been trained to a convergence state to generate a corresponding knowledge point label for each video key frame, wherein the knowledge point label represents a subject and its corresponding knowledge point category; specifically, the basic network architecture of the knowledge point classification model is a BERT model, and the subjects include Chinese, mathematics, physics, biology, fine arts, and music, etc. The knowledge point categories corresponding to the mathematics subjects include algebra, geometry, calculus, and statistics, etc., and the knowledge point categories corresponding to the physics subjects include mechanics, thermodynamics, electromagnetism, and optics, etc., wherein one subject corresponds to multiple knowledge point categories, and the knowledge point categories between different subjects are different.

[0122] For a specific embodiment, please refer to Figure 5 , the steps of training the knowledge point classification model include:

[0123] Step S401, obtaining a text sequence corresponding to any video key frame of an online teaching video in an online education platform as a training sample;

[0124] As mentioned above, the basic network architecture of the knowledge point classification model of the present application is the BERT model. The BERT model can be used to implement the text feature extractor in this embodiment. For the BERT model, it is necessary to implement training in advance and train it to a convergence state in order to efficiently serve the technical solutions of the various embodiments of the present application.

[0125] A better way is to regard the text sequence corresponding to any video key frame of the online teaching video in the online education platform as a training sample. Therefore, in this embodiment, the text sequence corresponding to any video key frame of the online teaching video in the online education platform can be obtained for training, wherein the text sequences corresponding to handwriting or texts with different fonts in lectures or teaching videos are all used as training samples, and the subject areas pointed to by the training samples and their corresponding knowledge point categories can be used as supervision labels after the BERT model is connected to the classifier.

[0126] Step S402: construct a coding vector from the training sample, call a text feature extractor in a preset knowledge point classification model, perform text feature extraction, and obtain corresponding text features;

[0127] The text sequence corresponding to any video key frame of the online teaching video in the online education platform is obtained, and the text sequence corresponding to any video key frame is constructed as a coding vector to meet the needs of BERT model input. The BERT model converts each word in the text sequence corresponding to any video key frame into a one-dimensional vector by querying the word vector table as the model input; in addition, in addition to the word vector, the model input also includes two other parts: a text vector and a position vector; the value of the text vector is automatically learned during the model training process, which is used to characterize the global semantic information of the text and merged with the semantic information of a single word / word; the position vector: since the semantic information carried by the words / words appearing at different positions in the text is different, for example, "I miss her" and "she misses me", the BERT model adds a different vector to the words / words at different positions for distinction; the BERT model uses the sum of the word vector, the text vector and the position vector as the coding vector required for the model input; the training sample can be pre-constructed as a corresponding coding vector to provide input for the BERT model; the BERT model extracts text features from the input coding vector according to its own implementation logic, and finally obtains the corresponding text features.

[0128] Step S403, calling a multi-layer fully connected neural network to classify each text feature to obtain a classification result, using the subject pointed to by the training sample and its corresponding knowledge point category as the supervision label of the classification result, and back-propagating to correct the weight parameters of the text feature extractor until its loss function converges to complete the training.

[0129] In order to realize the task training of the BERT model, the text features outputted by it are inputted into a multi-layer fully connected neural network for classification, so as to obtain the classification scores corresponding to each subject and its corresponding knowledge point category; since the subject and its corresponding knowledge point category are adopted as supervisory labels in the multi-layer fully connected neural network, the multi-layer fully connected neural network will back-propagate to the BERT model according to the supervision of the supervisory labels to correct its weight parameters until the entire loss function reaches convergence and finally completes the training. It can be understood that the loss function can be implemented based on the cross entropy loss function.

[0130] After the knowledge point classification model is trained to a convergent state, the text sequence corresponding to the video key frame is input into the knowledge point classification model that has been trained to a convergent state, so that the corresponding knowledge point label can be generated for each video key frame, and the corresponding subject and knowledge point category of each video key frame can be determined.

[0131] This embodiment implements the training process of the knowledge point classification model required by the technical solution of the present application. By training a knowledge point classification model, the corresponding knowledge point label is generated for each video key frame, and the subject and the corresponding knowledge point category corresponding to each video key frame are determined. It can be seen that since the text sequence corresponding to any video key frame of the online teaching video in the online education platform is used for training during the training process, the BERT model is easier to converge. The text feature extractor obtained thereby has a stronger representation learning ability and can more accurately match the text sequence corresponding to the video key frame to the appropriate subject and the corresponding knowledge point category, which helps to improve the content matching degree of the online teaching video and enhance the user interaction experience.

[0132] Step S50, based on the timestamps and knowledge point labels corresponding to the video key frames, the online teaching video is cut into multiple video clips, and the video clips corresponding to the same knowledge point label are merged to determine the online teaching video corresponding to each knowledge point label, so as to complete the classification extraction of knowledge points in the online teaching video.

[0133] The text sequence corresponding to the video key frame is input into the knowledge point classification model that has been trained to a convergence state. After generating the corresponding knowledge point label for each video key frame, the online teaching video is cut into multiple video clips based on the timestamps and knowledge point labels corresponding to the video key frames. The video clips corresponding to the same knowledge point label are merged to determine the online teaching video corresponding to each knowledge point label, so as to complete the classification extraction of knowledge points in the online teaching video.

[0134] For a specific embodiment, please refer to Figure 6 The steps of cutting the online teaching video into multiple video clips based on the timestamps and knowledge point labels corresponding to the video key frames, and merging the video clips corresponding to the same knowledge point label to determine the online teaching videos corresponding to the respective knowledge point labels include:

[0135] Step S501, obtaining video key frames in the online teaching video, and recording the timestamp of each video key frame in the online teaching video, wherein the timestamp is used to identify the precise position of each key frame;

[0136] Step S502: based on the timestamps and knowledge point tags corresponding to the video key frames, read the positions of the video key frames in the online teaching video to determine the video segmentation points;

[0137] Step S503, segmenting the online teaching video based on the video segmentation points to determine video segments corresponding to each knowledge point label, wherein the video segments correspond to the knowledge point labels one by one;

[0138] Step S504: merge the video clips corresponding to the same knowledge point label to determine the online teaching videos corresponding to the various knowledge point labels, so as to complete the classification extraction of knowledge points in the online teaching videos.

[0139] Specifically, the online teaching video management terminal can obtain the video key frames in the online teaching video and record the timestamp of each key frame in the video for subsequent processing and segmentation. The timestamp is used to identify the exact position of each key frame for use in the video segmentation process.

[0140] Furthermore, the timestamp information recorded in the above steps is used to read the positions of each key frame in the online teaching video to determine the segmentation point; a video editing tool (such as FFmpeg) can be used to segment the original video into multiple small video segments according to the timestamp information, wherein each small video segment corresponds to a specific knowledge point, for example, from time point A to time point B is a knowledge point segment, and from time point B to time point C is another knowledge point segment; the segmented small video segments are named and stored. Each video segment is named according to its corresponding knowledge point and stored in a preset folder, such as "Subject 1_Knowledge Point 1_Part 1.mp4", "Subject 1_Knowledge Point 1_Part 2.mp4", etc.

[0141] Arrange all relevant video segments according to knowledge points, call the preset video editing tool, merge all video segments belonging to the same knowledge point into a complete video file; store the merged complete video file in the specified directory for subsequent release and use. For example, merge "Subject 1_Knowledge Point 1_Part 1.mp4" and "Subject 1_Knowledge Point 1_Part 2.mp4" into "Subject 1_Knowledge Point 1_Complete Video.mp4" and store it in the "Subject 1_Knowledge Point 1" folder.

[0142] From the above steps, it can be seen that the subject and the corresponding knowledge point category corresponding to each video key frame in the online teaching video are determined through the knowledge point classification model, and the video clips corresponding to the same subject and the same knowledge point category are merged to determine the online teaching video corresponding to each knowledge point label. The educational video is efficiently processed and organized, and the complete video files of each knowledge point corresponding to each subject are output, which facilitates students to carry out targeted learning and improves the utilization rate of educational resources.

[0143] It can be seen from the above embodiments that, compared with the prior art, the present application aims at the processing of videos in the prior art, which has very high computational complexity and memory requirements, resulting in great limitations in real-time and efficiency, as well as poor effects when processing complex backgrounds and small-size texts, and inability to accurately identify detailed information. The present application includes but is not limited to the following beneficial effects:

[0144] First, the network teaching video knowledge point classification and extraction method proposed in this application is aimed at traditional teaching videos that require manual content annotation and classification, which is not only time-consuming and labor-intensive, but also prone to human errors and other problems; this application automatically extracts features and classifies knowledge points through video key frame recognition models, character recognition models, and knowledge point classification models, and can quickly and accurately annotate video content to improve the utilization efficiency of teaching resources; students and teachers can quickly find the network teaching videos corresponding to the required learning and teaching content through the classified knowledge points, saving time, improving the efficiency of learning and teaching, effectively improving the flexibility of online learning, and improving user efficiency;

[0145] Secondly, the video key frame recognition model proposed in this application adopts the deep learning model of attention mechanism for training, which greatly reduces the number of videos converted into images, relieves the calculation pressure of subsequent modules, and improves the overall operation efficiency; at the same time, the generated key frame images have low similarity and strong representativeness, which can be used as an overview of the key content of the video; through this method, ordinary computers can make full use of their computing power to run more advanced deep learning algorithms, significantly improving the performance of the video frame feature extraction algorithm;

[0146] Third, the character recognition model proposed in this application is constructed by a DB text detection algorithm module, a text direction classifier and a CRNN text recognition algorithm module. A text direction classifier is provided between the output of the DB text detection algorithm module and the input of the CRNN text recognition algorithm module to cope with text recognition in different directions and improve the accuracy of text recognition. At the same time, the CRNN text recognition algorithm is suitable for processing complex texts, and is suitable for complex and irregular texts, such as handwriting in lectures or teaching videos or texts with different fonts. It has strong robustness and is suitable for processing video key frames in course videos recorded during the teaching, academic lectures, academic conferences and scientific research training of this application, and classifying and extracting knowledge points of specific contents of the video key frames according to the recognition results;

[0147] Fourthly, this application realizes the accurate segmentation of different knowledge points extracted from long videos. This method ensures that the final generated video file has good playback continuity through optimized video merging technology, meeting the needs of actual teaching scenarios.

[0148] See also Figure 7, a network teaching video knowledge point classification and extraction device provided to meet one of the purposes of the present application, includes a teaching video acquisition module 1100, a key frame sequence construction module 1200, a text sequence generation module 1300, a knowledge point label generation module 1400 and a classification extraction module 1500. Among them, the teaching video acquisition module 1100 is configured to respond to an instruction to classify and extract knowledge points from network teaching videos, and obtain a network teaching video set, wherein the network teaching video set is constructed by a plurality of network teaching videos of different types; the key frame sequence construction module 1200 is configured to use a video key frame recognition model that has been trained to a convergence state to extract each video key frame from the video stream corresponding to the network teaching video to construct a video key frame sequence; the text sequence generation module 1300 is configured to use a preset character recognition model to extract text data of each video key frame in the video key frame sequence to generate text data corresponding to each video key frame. This sequence; a knowledge point label generation module 1400, which is configured to input the text sequence corresponding to the video key frame into a knowledge point classification model that has been trained to a convergence state, and generate a corresponding knowledge point label for each video key frame, wherein the knowledge point label represents a subject and its corresponding knowledge point category; a classification extraction module 1500, which is configured to cut the online teaching video into multiple video clips based on the timestamps and knowledge point labels corresponding to the video key frames, and merge the video clips corresponding to the same knowledge point label to determine the online teaching video corresponding to each knowledge point label, so as to complete the classification extraction of knowledge points in the online teaching video.

[0149] Based on any embodiment of this application, please refer to Figure 8 Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Figure 8 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a method for classifying and extracting knowledge points from online teaching videos. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the method for classifying and extracting knowledge points from online teaching videos of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0150] In this embodiment, the processor is used to execute Figure 7 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the network teaching video knowledge point classification and extraction device of this application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0151] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for classifying and extracting knowledge points from online teaching videos described in any embodiment of the present application.

[0152] The present application also provides a computer program product, including a computer program / instruction, which, when executed by one or more processors, implements the steps of the method for classifying and extracting knowledge points from online teaching videos described in any embodiment of the present application.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0154] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0155] To summarize, this application automatically performs feature extraction and knowledge point classification through video key frame recognition models, character recognition models, and knowledge point classification models, and can quickly and accurately annotate video content, thereby improving the utilization efficiency of teaching resources; students and teachers can use the classified knowledge points to quickly find online teaching videos corresponding to the required learning and teaching content, saving time, improving learning and teaching efficiency, effectively improving the flexibility of online learning, and improving user efficiency.

Claims

1. A method for classifying and extracting knowledge points from online teaching videos, characterized in that: include: In response to an instruction to classify and extract knowledge points from online teaching videos, a collection of online teaching videos is obtained, wherein the collection of online teaching videos is constructed from a plurality of online teaching videos of different types; Using a video key frame recognition model that has been trained to a convergence state, extracting each video key frame from a video stream corresponding to the online teaching video to construct a video key frame sequence; Extracting text data of each video key frame in the video key frame sequence using a preset character recognition model to generate a text sequence corresponding to each video key frame; Inputting the text sequence corresponding to the video key frame into the knowledge point classification model that has been trained to a convergence state, generating a corresponding knowledge point label for each video key frame, wherein the knowledge point label represents the subject and its corresponding knowledge point category; Based on the timestamps and knowledge point labels corresponding to the video key frames, the online teaching video is cut into multiple video clips, and the video clips corresponding to the same knowledge point label are merged to determine the online teaching video corresponding to each knowledge point label, so as to complete the classification extraction of knowledge points in the online teaching video.

2. The method for classifying and extracting knowledge points from online teaching videos according to claim 1, characterized in that: Before the step of extracting each video key frame from the video stream corresponding to the online teaching video using the video key frame recognition model that has been trained to a convergence state to construct a video key frame sequence, the method includes: In response to the preprocessing instruction, an original frame rate of a video stream corresponding to the online teaching video is obtained, wherein the original frame rate represents the number of image frames per second; Based on the original frame rate of the video stream corresponding to the online teaching video, video image frames in the video stream corresponding to the online teaching video are extracted at preset time intervals as target video image frames of the online teaching video.

3. The method for classifying and extracting knowledge points from online teaching videos according to claim 1, characterized in that: The step of extracting each video key frame from the video stream corresponding to the online teaching video using the video key frame recognition model that has been trained to a convergence state to construct a video key frame sequence includes: Determine a target video image frame of the online teaching video, and input the target video image frame into a video key frame recognition model that has been trained to a convergence state; Perform image block embedding on the input target video image frame to obtain multiple image block vectors, and add a classification vector to form multiple embedding vectors; Adding a position encoding vector to the multiple embedding vectors to form an input vector, wherein the position encoding vector is used to maintain spatial position information between image blocks; Stacking a plurality of encoding modules for the input vector to perform feature extraction to determine a feature vector corresponding to each target video image frame, wherein the encoding module includes a plurality of multi-head self-attention encoding layers; The Euclidean distance between the feature vectors of adjacent video image frames in the target video image frame is calculated and determined, and whether the Euclidean distance exceeds a preset change rate threshold is detected. If exceeded, the target video image frame exceeding the preset change rate threshold is used as a video key frame to construct a video key frame sequence.

4. The method for classifying and extracting knowledge points from online teaching videos according to claim 1, characterized in that: The step of extracting text data of each video key frame in the key video image sequence by using a preset character recognition model to generate a text sequence corresponding to each video key frame comprises: Get the video key frame containing the characters to be recognized; Input the video key frame containing the characters to be recognized into a preset character recognition model, wherein the character recognition model is constructed by a DB text detection algorithm module, a text direction classifier and a CRNN text recognition algorithm module, and a text direction classifier is provided between the output of the DB text detection algorithm module and the input of the CRNN text recognition algorithm module; The key frame sequence number and text data corresponding to the video key frame are output to generate a text sequence corresponding to each video key frame.

5. The method for classifying and extracting knowledge points from online teaching videos according to claim 1, characterized in that: The steps for training the knowledge point classification model include: Obtaining a text sequence corresponding to any video key frame of an online teaching video in an online education platform as a training sample; Constructing a coding vector from the training sample, calling a text feature extractor in a preset knowledge point classification model to extract text features and obtain corresponding text features; A multi-layer fully connected neural network is called to classify each text feature to obtain the classification result. The subject pointed to by the training sample and its corresponding knowledge point category are used as the supervision label of the classification result. The weight parameters of the text feature extractor are corrected by backpropagation until its loss function converges to complete the training.

6. The method for classifying and extracting knowledge points from online teaching videos according to claim 1, characterized in that: The steps of cutting the online teaching video into multiple video clips based on the timestamps and knowledge point labels corresponding to the video key frames, and merging the video clips corresponding to the same knowledge point label to determine the online teaching videos corresponding to the respective knowledge point labels include: Obtaining video key frames in the online teaching video, and recording the timestamp of each video key frame in the online teaching video, wherein the timestamp is used to identify the precise position of each key frame; Based on the timestamps and knowledge point labels corresponding to the video key frames, the positions of the video key frames in the online teaching video are read to determine the video segmentation points; Segmenting the online teaching video based on the video segmentation points to determine video segments corresponding to each knowledge point label, wherein the video segments correspond to the knowledge point labels one by one; The video clips corresponding to the same knowledge point label are merged to determine the online teaching videos corresponding to each knowledge point label, so as to complete the classification extraction of knowledge points in the online teaching videos.

7. The method for classifying and extracting knowledge points from online teaching videos according to any one of claims 1 to 6, characterized in that: The basic network architecture of the video key frame recognition model is the ViT model; the basic network architecture of the knowledge point classification model is the BERT model.

8. A device for classifying and extracting knowledge points from online teaching videos, characterized in that: include: A teaching video acquisition module, configured to respond to an instruction to classify and extract knowledge points from online teaching videos, and acquire an online teaching video set, wherein the online teaching video set is constructed from a plurality of online teaching videos of different types; A key frame sequence construction module is configured to extract each video key frame from the video stream corresponding to the online teaching video using a video key frame recognition model that has been trained to a convergence state to construct a video key frame sequence; A text sequence generation module is configured to extract text data of each video key frame in the video key frame sequence using a preset character recognition model to generate a text sequence corresponding to each video key frame; A knowledge point label generation module is configured to input the text sequence corresponding to the video key frame into the knowledge point classification model that has been trained to a convergence state, and generate a corresponding knowledge point label for each video key frame, wherein the knowledge point label represents a subject and its corresponding knowledge point category; The classification extraction module is configured to cut the online teaching video into multiple video clips based on the timestamps and knowledge point labels corresponding to the video key frames, merge the video clips corresponding to the same knowledge point labels to determine the online teaching videos corresponding to the various knowledge point labels, so as to complete the classification extraction of knowledge points in the online teaching video.

9. An electronic device, comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images