A method for determining target video attribute characteristics, storage medium, and electronic device

Through gesture detection neural network, the gesture labels of teachers in online education are identified, the number of meaningful gestures appears, and the teaching quality is evaluated. The problem of manual monitoring of classes cannot be discovered in a timely manner, and the timely screening and improvement of teaching quality is achieved.

CN112270231BActive Publication Date: 2024-11-15BEIJING DA MI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011120702.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-10-19
Publication Date
2024-11-15
Estimated Expiration
2040-10-19

AI Technical Summary

Technical Problem

In online education, due to the surge in the number of courses, manual supervision cannot detect negative teaching courses in a timely manner, which may cause adverse effects.

Method used

By obtaining the image collection of target videos, using pre-trained gesture detection neural network to identify the teacher's gesture tags, calculate the number of occurrences of meaningful gestures in unit time, determine the label characteristics and attribute characteristics of the video, thereby evaluating teaching quality and identifying Negative teaching courses.

Benefits of technology

It has achieved timely detection of negative teaching courses, reduced adverse effects on students, and improved the teaching quality and reputation of online education.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112270231B_ABST
    Figure CN112270231B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a method, storage medium and electronic device for determining the attribute characteristics of a target video. By acquiring a target video of online teaching, an image set is acquired according to the target video, and the image set is input into a gesture detection neural network as a detection set, the gesture labels of the teacher in the classroom teaching are detected, and whether the gestures used by the teacher in the classroom are meaningful is determined according to the gesture labels, and the teaching quality of the online teaching is determined according to the number of times the meaningful gesture labels appear in a unit time, and an evaluation result is given; thereby, the performance of the teacher in the classroom can be identified, and negative courses with poor teaching quality can be screened out in time to avoid adverse effects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of online education, and in particular to a method for determining attribute features of a target video. Background Art

[0002] In order to improve the teaching quality during online teaching, it is necessary to monitor the teaching performance of classroom teachers, that is, to supervise the class. In the existing technology, manual supervision is usually used to monitor the classroom performance of teachers. With the development of online education, the number of courses has increased dramatically, and manual supervision cannot meet the requirements. Failure to discover negative teaching courses in time may cause adverse effects. Summary of the invention

[0003] In view of this, in order to timely discover negative teaching courses and avoid adverse effects, the embodiments of the present invention provide a method, a storage medium, and an electronic device for determining attribute features of a target video.

[0004] In a first aspect, an embodiment of the present invention provides a method for determining a target video attribute feature, comprising:

[0005] Acquire an image set according to a target video, wherein the image set includes a plurality of images, and the images include a target person;

[0006] Recognize each image in the image set by using a pre-trained gesture detection neural network to obtain a gesture label of a target person in each image;

[0007] Determining a label feature of the target video according to the gesture label of each image in the image set;

[0008] An attribute feature of the label feature in the target video is determined, where the attribute feature is used to characterize the performance of the target task in the target video.

[0009] Preferably, the gesture detection neural network is trained by the following steps:

[0010] Obtain each training sample in the training sample set;

[0011] Get the initialized gesture detection neural network;

[0012] The parameters of the gesture detection neural network are adjusted according to the training samples to extract feature information through a predetermined loss function, and in response to the feature information satisfying a verification condition, the trained gesture detection neural network is determined.

[0013] Preferably, the gesture detection neural network comprises a glance neural network.

[0014] Preferably, the characteristic information includes:

[0015] Prediction box location information and confidence;

[0016] The prediction box position information includes positioning information and aspect ratio information, and the aspect ratio information is pre-set or determined by clustering pictures of the target object.

[0017] Preferably, the step of training the gesture detection neural network further comprises:

[0018] Expand the training sample set through image enhancement;

[0019] Wherein, the expanding the training sample set by image enhancement includes:

[0020] Mirroring the training sample image to obtain an expanded training sample image; and / or

[0021] The training sample image is rotated to obtain an expanded training sample image.

[0022] Preferably, the predetermined loss function is a focal loss function.

[0023] Preferably, the method further comprises:

[0024] Obtain the confidence and intersection-over-union ratio corresponding to the target image through a predetermined loss function;

[0025] The label is determined according to the confidence and the intersection-over-union ratio.

[0026] Preferably, the label feature of the target video is obtained by counting the number of times the meaningful gesture appears in the target video within a predetermined time;

[0027] The attribute feature is obtained through the label feature of the target video, and is used to characterize the performance of the target person in the target video; in response to the label feature being high quality, the attribute feature is that the performance of the target person in the target video provides a high-quality teaching course for the target person; in response to the label feature being negative, the attribute feature is that the performance of the target person in the target video provides a negative teaching course for the target person.

[0028] In a second aspect, an embodiment of the present invention provides a computer-readable storage medium for storing computer program instructions, wherein the computer program instructions implement any of the above-described methods when executed by a processor.

[0029] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, characterized in that the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement any of the methods described above.

[0030] The embodiment of the present invention provides a method, storage medium and electronic device for determining the attribute characteristics of a target video. By acquiring a target video of online teaching, an image set is acquired according to the target video, and the image set is input into a gesture detection neural network as a detection set, the gesture labels of the teacher in the classroom teaching are detected, and whether the gestures used by the teacher in the classroom are meaningful is determined according to the gesture labels, and the teaching quality of the online teaching is determined according to the number of times the meaningful gesture labels appear in a unit time, and an evaluation result is given; thereby, the performance of the teacher in the classroom can be identified, and negative courses with poor teaching quality can be screened out in time to avoid adverse effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0032] Figure 1 is a flow chart of a method for determining target video attribute characteristics according to an embodiment of the present invention;

[0033] Figure 2 is a schematic diagram of a gesture recognition tag according to an embodiment of the present invention;

[0034] Figure 3 is a schematic diagram of label features of a target video according to an embodiment of the present invention;

[0035] Figure 4 is a schematic diagram of label features of a target video according to an embodiment of the present invention;

[0036] Figure 5 is a flow chart of training a gesture detection neural network according to an embodiment of the present invention;

[0037] Figure 6 is a schematic diagram of a prediction frame according to an embodiment of the present invention;

[0038] Figure 7 is a schematic diagram of an online education class monitoring system according to an embodiment of the present invention;

[0039] Figure 8 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the detailed description of the present invention below, some specific details are described in detail. It is possible for a person skilled in the art to fully understand the present invention without the description of these details. In order to avoid confusing the essence of the present invention, known methods, processes, flows, components and circuits are not described in detail.

[0041] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0042] Unless the context clearly requires otherwise, words such as “include”, “including” and the like in the specification should be interpreted as including rather than being exclusive or exhaustive; that is, as meaning “including but not limited to”.

[0043] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0044] Currently, the teaching quality of online education teachers is mainly supervised by manual supervision. High-quality teaching courses can benefit students and bring better reputation to the platform; negative teaching courses will have a bad influence on students, which in turn affects the reputation of the platform and even bring bad social impact.

[0045] With the development of online education, the number of courses has increased dramatically, and manual supervision cannot meet the requirements. If negative teaching courses are not discovered in time, adverse effects may occur.

[0046] High-quality online education courses often have the following characteristics: teachers tend to use more meaningful gestures; on the contrary, in negative online education courses, teachers use significantly fewer meaningful gestures.

[0047] Specifically, the meaningful gestures used by teachers in teaching courses include: instruction gestures, interactive gestures, teaching gestures, etc.

[0048] Further subdivided, the command gestures include: one hand opened to the ear to indicate listening, one hand placed horizontally across the eyebrows to indicate looking, and other similar command gestures.

[0049] Interactive gestures include: five fingers spread out to indicate a job well done, clapping hands to indicate encouragement to students, and other similar interactive gestures.

[0050] Teaching gestures include: teachers holding teaching aids, holding a whiteboard, and other similar teaching gestures.

[0051] Specifically, refer to Table 1.

[0052]

[0053]

[0054] Table 1 Meaningful gestures

[0055] The embodiment of the present invention records the teaching course of the teacher as a target video, identifies the gesture label of the teacher through the gesture detection neural network, identifies the meaningful gestures used by the teacher in the course based on the gesture labels, and determines the label features of the target video by counting the number of times the meaningful gestures appear in a predetermined time period.

[0056] The embodiment of the present invention divides the label features of the target video into high-quality and negative.

[0057] In the target video, if the number of meaningful gestures that appear every 3 minutes reaches 50-55 times, the label feature of the target video is defined as high quality. In such a target video, the teacher interacts with the students many times, the students receive more encouragement from the teacher, and the teacher gives clear teaching instructions through gestures.

[0058] In another case, if meaningful gestures appear only 5-10 times every 3 minutes in the target video, the label feature in the target video is defined as negative. In such a target video, the teacher rarely interacts with the students, the students receive little encouragement, and the teacher does not give clear teaching instructions through gestures.

[0059] Next, the attribute features of the target video are determined through the label features of the target video, and the attribute features can represent the performance of the teacher in the classroom.

[0060] If the label feature is high quality, the attribute feature of the target video is: the teacher provides high-quality teaching courses.

[0061] If the label feature is negative, the attribute feature of the target video is: the teacher provides negative teaching courses (usually also called gray courses).

[0062] The embodiment of the present invention can reduce the negative impact on students by timely discovering the negative teaching courses provided by the teacher.

[0063] Figure 1 The present invention is a flowchart of a method for determining target video attribute characteristics according to an embodiment of the present invention.

[0064] Reference Figure 1 , the method for determining the attribute characteristics of a target video according to an embodiment of the present invention includes the following steps.

[0065] Step 100: acquiring an image set according to a target video, wherein the image set includes a plurality of images, and the images include a target person.

[0066] In the embodiment of the present invention, the entire process of a teacher teaching students is recorded as a target video.

[0067] The target video is converted into an image set by a video conversion tool, and the image set includes multiple images, and the images can show the performance of the teacher in the classroom. In an optional implementation, FFmpeg (Fast ForwardMoving Picture Experts Group, a multimedia processing tool) is used to convert the target video into an image set. FFmpeg is an open source computer program that can be used to record, convert digital audio and video, and convert them into streams. It provides a complete solution for recording, converting and streaming audio and video.

[0068] Each image in the image set can show the teacher's expression, body movements and gestures in the classroom. Taking online education as an example, the teacher's expression, body movements and gestures in the classroom can reflect the attractiveness of the course to students. The corresponding targets can be identified by different types of neural networks. The embodiment of the present invention is described by taking the recognition of the teacher's gesture as a preferred embodiment.

[0069] In order to identify the teacher's gesture, the embodiment of the present invention identifies the teacher's gesture label in each image through a gesture detection neural network.

[0070] Step 200: Recognize each image in the image set by using a pre-trained gesture detection neural network to obtain a gesture label of a target person in each image.

[0071] In online teaching courses, teachers are used as target characters. The image set containing teachers' performance is used as the detection set, and the detection set is input into the gesture detection neural network to obtain the gesture label of the teacher in each image.

[0072] Specific reference Figure 2 , Figure 2 is a schematic diagram of a gesture recognition tag according to an embodiment of the present invention.

[0073] An image of a teacher’s hand behind his ear is fed into a gesture detection neural network.

[0074] This image is recognized through a gesture detection neural network, which identifies the teacher's gesture label in the image as "one hand open to the ear to indicate listening."

[0075] Next, the gesture label is extracted to obtain the teacher's gesture. The teacher's gesture in the image is "listening".

[0076] That is to say, if the teacher in the image uses meaningful gestures, the gesture detection neural network outputs a gesture label; if the teacher in the image does not use meaningful gestures, the gesture detection neural network does not output a gesture label.

[0077] Specifically, meaningful gestures can show the interaction between teachers and students, teachers' encouragement to students, and teachers' clear teaching instructions. Meaningful gestures can also be subdivided into instruction gestures, interactive gestures, teaching gestures, etc.

[0078] In order to determine the teaching quality of the teacher in the target video, it is necessary to determine the label features of the target video according to the characteristics of meaningful gestures appearing in the target video.

[0079] Step 300: determining a label feature of the target video according to the gesture label of each image in the image set.

[0080] Specific reference Figure 3 and Figure 4 , Figure 3 is a schematic diagram of label features of a target video according to an embodiment of the present invention; Figure 4 Schematic diagram of label features of a target video according to an embodiment of the present invention.

[0081] Figure 3 and Figure 4 The horizontal axis represents the number of times a meaningful gesture occurs, and the unit of the horizontal axis is the number of times. The vertical axis represents the time interval, and the unit of the vertical axis is minutes.

[0082] from Figure 3 It can be seen that if the number of meaningful gestures in the target video reaches 50-55 times every 3 minutes, in such a target video, the teacher interacts with the students many times, the students receive more encouragement from the teacher, and the teacher gives clear teaching instructions through gestures. Therefore, the label features of the target video that meets such characteristics are defined as high quality.

[0083] from Figure 4 It can be seen from the figure that if the meaningful gestures in the target video appear only 5-10 times every 3 minutes, in such a target video, the teacher rarely interacts with the students, the students receive little encouragement, and the teacher does not give clear teaching instructions through gestures. The label features of the target video that meets such characteristics are defined as negative.

[0084] That is, the label feature of the target video can be obtained by counting the number of times the meaningful gestures appear in the target video within a predetermined time.

[0085] In order to determine the teacher's performance in the classroom in the target video, next, it is necessary to determine the attribute characteristics of the target video.

[0086] Step 400: determining the attribute features of the label features in the target video, wherein the attribute features are used to characterize the performance of the target task in the target video.

[0087] The embodiment of the present invention defines the performance of the teacher in the classroom through attribute features.

[0088] The attribute features are obtained through the label features of the target video.

[0089] Specifically, the label feature of the target video is obtained, and if the label feature is high-quality, the teacher's performance in the classroom is defined as the teacher providing high-quality teaching courses. That is, the attribute feature is: the teacher provides high-quality teaching courses.

[0090] In another case, if the label feature is negative, the teacher's performance in the classroom is defined as the teacher providing a negative teaching course (usually also called a gray course). That is, the attribute feature is: the teacher provides a negative teaching course.

[0091] By timely discovering the attribute characteristics of the target video, negative teaching courses can be discovered in time, thereby reducing the adverse effects on students.

[0092] Before determining the target video attribute feature in the embodiment of the present invention, it is necessary to pre-train the gesture detection neural network.

[0093] Training a gesture detection neural network requires a large number of images, which are usually divided into training sample images and verification images according to a certain ratio. For example, 70%-80% of the images are divided into the training sample set, and 30%-20% of the images are divided into the verification set.

[0094] The more sufficient the number of training sample images in the training sample set is, the better the training effect is, and the more accurate the gestures of the target person are recognized.

[0095] However, the number of training sample images for some gestures is small. In order to achieve better training results, it is necessary to expand the training sample set through image enhancement.

[0096] Preferably, the embodiment of the present invention expands the training sample image by mirroring the training sample image; and / or expands the training sample image by rotating the training sample image. Specifically, multiple images are generated from one image by mirroring the training sample image left and right, mirroring the training sample image up and down, etc. Alternatively, multiple images are generated from one image by rotating the training sample image at various angles.

[0097] It is also possible to generate multiple images from one image through processing methods such as cropping, translation, interpolation, Gaussian noise, contrast transformation, etc., thereby expanding the training samples.

[0098] In order to obtain a trained gesture detection neural network, the gesture detection neural network in the embodiment of the present invention is obtained through the following steps.

[0099] Figure 5 4 is a flow chart of training a gesture detection neural network according to an embodiment of the present invention.

[0100] Reference Figure 5 , step 500, obtaining each training sample in the training sample set.

[0101] The training samples are images including the gestures of the target person. A large number of training samples constitute the training sample set.

[0102] The embodiment of the present invention collects a large number of images including gestures of a target person to form a training sample set.

[0103] Step 600, obtaining an initialized gesture detection neural network.

[0104] The initial model of the gesture detection neural network in the embodiment of the present invention is a YOLO V3 neural network (YOLO, You Only Look Once, V3 is the third version).

[0105] The YOLO V3 neural network can solve both image recognition (also commonly known as regression) and classification problems.

[0106] That is to say, after receiving a given image, the YOLO V3 neural network can identify the location of the target person's gesture in the image and give a classification of the target person's gesture.

[0107] Specifically, the YOLO V3 neural network can determine the prediction box (bounding box) of the image. The prediction box has feature information, and the position of the target person's gesture in the image and the classification of the target person's gesture can be obtained based on the feature information.

[0108] Step 700, adjusting various parameters of the gesture detection neural network according to the various training samples to extract feature information through a predetermined loss function, and in response to the feature information satisfying a verification condition, determining it as a trained gesture detection neural network.

[0109] The embodiment of the present invention uses a focus loss function (FOCAL LOSS) in the process of training the gesture detection network. The focus loss function is modified based on the standard cross entropy loss function.

[0110] In the sample set of the embodiment of the present invention, the number of training sample images of some gestures is sufficient, while the number of training sample images of other gestures is less. If the standard cross entropy loss function is used to classify the images, the accuracy is not high. The focus loss function can reduce the weight of such training samples with sufficient training samples, so that a better classification effect can be achieved, and the gesture labels obtained are more accurate. In other words, the focus loss function can achieve the same recognition effect and improve the accuracy of classification.

[0111] Through the focal loss function, the prediction box of the target person's gesture can be identified at multiple locations in a given image. For example, if the given image is an image of a teacher with one hand behind his ear, the prediction box of the target person's gesture can be identified at multiple locations in the given image.

[0112] The prediction box has feature information, including the prediction box location information and confidence.

[0113] Among them, the prediction box position information includes positioning information and aspect ratio information (anchor box). Since the gesture of the target person has certain characteristics, appropriate aspect ratio information is conducive to the prediction box's recognition of the gesture. The aspect ratio information is pre-set or determined by clustering the image set of the target object. Specifically, the aspect ratio information can be set by an empirical value; it can also be obtained by clustering the image set of the target object.

[0114] Figure 6 is a schematic diagram of a prediction frame according to an embodiment of the present invention.

[0115] Reference Figure 6 , taking the training sample as an image of a teacher with one hand behind his ear as an example to illustrate the prediction frame. The prediction frames obtained by the focal loss function are: the first prediction frame 1, the second prediction frame 2 and the third prediction frame 3.

[0116] Those skilled in the art should understand that the number of prediction boxes is related to the configuration parameters of the computer hardware and the speed at which the gesture detection neural network is expected to respond. In theory, there is no limit to the number of prediction boxes. The embodiment of the present invention only uses three prediction boxes for illustration. The number of prediction boxes in practice can be determined based on the usage scenario.

[0117] In order to determine the final prediction box and classification of the target person's gesture. Specifically, the final prediction box and classification of the target person's gesture can be determined according to the confidence. In an optional implementation, the final prediction box and classification of the target person's gesture are determined by non-maximum suppression (NMS), and the specific steps are as follows:

[0118] Step S1: Sort the confidence scores of all prediction boxes in descending order;

[0119] Step S2: Select the prediction box P with the highest confidence and calculate the intersection over union (IOU) between P and other prediction boxes.

[0120] Step S3: Get each IOU, delete the prediction box whose IOU is greater than the predetermined value, and only keep the prediction box whose IOU is less than the predetermined value;

[0121] Step S4: Return to S1 until only one prediction box remains as the final prediction box of the target person's gesture, and the classification of this prediction box is used as the classification of the target person's gesture.

[0122] The position information and confidence of the final prediction box are obtained as the output result of the training set. The output result of the training set is compared with the data of the verification set. When the error between the two meets the predetermined condition, it is determined to be a trained gesture detection neural network.

[0123] For the case where the gap between some training samples is relatively large, for example, the gap between the training samples of "other meaningful gestures" in Table 1 is relatively large, the steps for obtaining the final prediction box of such samples are changed to: filter the label of the target object by reaching the predetermined value of the confidence and intersection-over-union ratio of the target object. The specific steps include the following:

[0124] Step S1: Sort the confidence scores of all prediction boxes in descending order;

[0125] Step S2: Filter out prediction boxes with confidence greater than 0.5 and IOU greater than 0.3;

[0126] Step S3: Determine the gesture as meaningful.

[0127] Figure 7 Schematic diagram of an online education class monitoring system according to an embodiment of the present invention.

[0128] The online education class monitoring system 70 of the embodiment of the present invention is run on the server side.

[0129] Through the experience of manual class monitoring, it is found that in online education classes, if the teacher has rich meaningful gestures, the student experience is better; on the contrary, in some classes, the teacher has fewer meaningful gestures, in which case the student experience is often poor, and such classes are defined as gray classes. Classes with fewer meaningful gestures (i.e., gray classes) are identified to avoid negative effects on students.

[0130] To this end, the embodiment of the present invention expects to recognize meaningful gestures through the gesture detection neural network. Specifically, the initial model of the gesture detection neural network in the embodiment of the present invention is the YOLO V3 neural network.

[0131] First, the initial model YOLO V3 is trained. Before training, a training sample set and a validation set are prepared. The training sample set and the validation set are a large number of pictures containing the teacher's gestures.

[0132] The training samples are input into the initial model YOLO V3, and the initial model YOLO V3 recognizes the samples. The recognition results are compared with the data in the validation set. If the recognition requirements are not met, the parameters in YOLO V3 are adjusted. When the recognition requirements are met, the gesture detection neural network is obtained.

[0133] Next, the online education class monitoring system 70 identifies the classroom video file and gives the evaluation result of the classroom video. The online education class monitoring system 70 includes: an acquisition module 71, an identification module 72, an extraction module 73 and an evaluation module 74.

[0134] The acquisition module 71 acquires the classroom video file and obtains an image set by sampling and extracting frames through FFmpeg.

[0135] Specifically, classroom video files can be obtained through the server.

[0136] When the teacher is teaching students, the teacher's image in the classroom is transmitted back to the server through the teacher terminal. The server records the teacher's image in the classroom as a classroom video file.

[0137] Specifically, the server samples the classroom video file through FFmpeg and converts the classroom video file into an image set, wherein the images in the image set include the teacher's performance in the classroom, such as facial expressions, body movements and gestures.

[0138] The recognition module 72 recognizes meaningful gestures in the image collection.

[0139] The gesture detection neural network recognizes gesture labels in images. When a picture is input into the gesture detection neural network, if the picture contains meaningful gestures, the gesture detection neural network will give a gesture label; if the picture does not contain meaningful gestures, the gesture detection neural network will not give a gesture label.

[0140] The extraction module 73 extracts gesture tags and summarizes gesture data.

[0141] If the gesture detection neural network gives a picture of a gesture label, extract the recognized gesture label. Each time a gesture label is extracted, the accumulated value is increased by 1.

[0142] Summarize gesture data and summarize the cumulative values ​​of gesture tags that appear every 3 minutes.

[0143] Online education classes usually last more than 20 minutes. In classes where students have a good experience, meaningful gestures usually appear 50-55 times every three minutes, with a minimum of 7 times and a maximum of 103 times.

[0144] In classes where students had a poor experience, meaningful gestures usually appeared 5-10 times every three minutes, with a minimum of 0 times and a maximum of 104 times every three minutes.

[0145] The evaluation module 74 evaluates the teaching quality and provides the evaluation results.

[0146] The number of meaningful gestures per unit time can be used to identify courses where the teacher is negative and evaluate such courses as gray courses. Specifically, a threshold can be set to identify courses where the teacher is negative and evaluate such courses as gray courses.

[0147] Based on the evaluation results, gray courses can be discovered in a timely manner, that is, courses that may have a negative impact on students can be discovered in a timely manner.

[0148] The embodiment of the present invention provides a method for determining the attribute features of a target video, by acquiring a target video for online teaching, acquiring an image set according to the target video, inputting the image set as a detection set into a gesture detection neural network, detecting the gesture labels of the teacher in the classroom teaching, determining whether the gestures used by the teacher in the classroom are meaningful according to the gesture labels, determining the teaching quality of the online teaching according to the number of occurrences of the meaningful gesture labels per unit time, and providing an evaluation result; thereby, the teacher's performance in the classroom can be identified, and negative courses with poor teaching quality can be screened out in time to avoid causing adverse effects.

[0149] Figure 8 is a schematic diagram of an electronic device according to an embodiment of the present invention.

[0150] like Figure 8The electronic device 8 shown includes a general hardware structure, which at least includes a processor 81 and a memory 82. The processor 81 and the memory 82 are connected via a bus 83. The memory 82 is suitable for storing instructions or programs executable by the processor 81. The processor 81 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 81 executes the instructions stored in the memory 82, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 83 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 84 and the display device and the input / output (I / O) device 85. The input / output (I / O) device 85 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 85 is connected to the system via an input / output (I / O) controller 86.

[0151] It should be understood by those skilled in the art that the embodiments of the present application may provide methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0153] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0154] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0155] Another embodiment of the present invention relates to a non-volatile readable storage medium for storing a computer-readable program, wherein the computer-readable program is used to enable a computer to execute some or all of the above method embodiments.

[0156] That is, those skilled in the art can understand that all or part of the steps in the method for implementing the above embodiments can be completed by specifying relevant hardware through a program, and the program is stored in a readable storage medium, including a number of instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0157] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for determining target video attribute characteristics, characterized in that: The method comprises: Acquire an image set according to a target video, wherein the image set includes a plurality of images, and the images include a target person; Recognize each image in the image set by using a pre-trained gesture detection neural network to obtain a gesture label of a target person in each image; Determining a label feature of the target video according to the gesture label of each image in the image set; Determining an attribute feature of the tag feature in a target video, wherein the attribute feature is used to characterize the performance of the target person in the target video; The step of determining the label feature of the target video according to the gesture label of each image in the image set includes: The label feature of the target video can be obtained by counting the number of times the meaningful gestures appear in the target video within a predetermined time, wherein the meaningful gestures include instruction gestures, interactive gestures and teaching gestures, and the label feature of the target video is high quality or negative; The gesture detection neural network is trained by the following steps: Obtain each training sample in the training sample set; Obtaining an initialized gesture detection neural network, wherein the initialized gesture detection neural network is a YOLO neural network; According to the training samples, the parameters of the gesture detection neural network are adjusted to identify the prediction frame of the target person's gesture through a predetermined loss function, and feature information of the prediction frame is extracted, wherein the feature information includes prediction frame position information and confidence, and in response to the feature information satisfying the verification condition, the trained gesture detection neural network is determined; Wherein, in response to the feature information satisfying the verification condition, determining that the gesture detection neural network is trained includes: Obtain the location information and corresponding confidence of the final prediction box as the output result of the training sample set; Comparing the output result with the data of the verification set, and determining it as a trained gesture detection neural network when the error between the two meets a predetermined condition; The final prediction frame and classification of the target person's gesture are determined by the following steps: Sort the confidence of all prediction boxes in descending order; Select the prediction box with the highest confidence and calculate the intersection and union ratio of the highest prediction box and other prediction boxes; Obtain each IoU ratio, delete the prediction frames whose IoU ratio is greater than a predetermined value, and only retain the prediction frames whose IoU ratio is less than the predetermined value; Then sort the confidence scores of all prediction boxes in descending order until only one prediction box remains as the final prediction box of the target person's gesture, and the classification of this prediction box is used as the classification of the target person's gesture.

2. The method according to claim 1, characterized in that The prediction box position information includes positioning information and aspect ratio information, and the aspect ratio information is preset or determined by clustering pictures of the target object.

3. The method according to claim 2, characterized in that The step of training the gesture detection neural network also includes: Expand the training sample set through image enhancement; Wherein, the expanding the training sample set by image enhancement includes: Mirroring the training sample image to obtain an expanded training sample image; and / or The training sample image is rotated to obtain an expanded training sample image.

4. The method according to claim 1, characterized in that: The predetermined loss function is a focal loss function.

5. The method according to claim 1, characterized in that The attribute feature is obtained through the label feature of the target video and is used to characterize the performance of the target person in the target video; in response to the label feature being high-quality, the attribute feature provides a high-quality teaching course for the target person for the performance of the target person in the target video; in response to the label feature being negative, the attribute feature provides a negative teaching course for the target person for the performance of the target person in the target video.

6. A computer-readable storage medium for storing computer program instructions, characterized in that: The computer program instructions implement the method according to any one of claims 1 to 5 when executed by a processor.

7. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 5.