Courseware identification-based video outline generation method, apparatus and device, and medium

Abstract

The embodiment of the invention discloses a courseware identification-based video outline generation method and device, electronic equipment and a storage medium. By recording the data source and the change event which cause the change of the main display content in the classroom video, confirming the occurrence time of the change event and the core content of the page after the occurrence, and taking the core content as the outline information of the corresponding page display time period, the content of the classroom video can be displayed without identifying the content of the classroom video. Through parallel recording of the two paths of associated information, one path of information with simple processing requirements is processed, so that accurate outline information of one path of information of which content structure analysis can be realized only through complex processing can be obtained, and accurate structure division and outline information identification of classroom videos based on teaching contents are realized.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of educational multimedia technology, and in particular to a method, device, equipment and medium for generating a video outline based on courseware recognition. Background Art

[0002] With the continuous development and popularization of information technology, in the field of teaching, computer-assisted teaching, distance learning, autonomous online learning, electronic classrooms and other forms of education based on information technology have made great progress. Classroom video recording has become the norm in the modern teaching process. Recorded classroom videos can be used for students' distance learning, after-class learning, teachers' after-class review and teaching reflection, and play a more obvious role in achieving educational equity and improving teaching standards.

[0003] Classroom videos are usually long videos of about 40 minutes. Students who did not participate in the classroom recording usually need to watch the video carefully to roughly understand the content of the class. When students who participated in the classroom recording need to review certain knowledge points that they did not understand or were interested in after class, they also need to use the progress bar to constantly search and locate them. When teachers who participated in the classroom recording want to reflect on their teaching after class, they need to use the video progress bar to constantly jump back and forth to retrieve and locate specific teaching clips. In order to achieve rapid positioning, classroom videos are usually manually added or the outline of the classroom video is added after the video content is recognized by algorithms.

[0004] When the inventor studied the existing solutions for automatically dividing video segments in non-educational scenarios and automatically identifying the video outlines of the corresponding segments, he found that the existing solutions for automatically identifying video outlines usually require a certain amount of video annotation data to train the model, and the model is used to identify the video subtitle content and the picture to complete the division. Since the boundaries of video segments are usually fuzzy, the existing solutions for automatically identifying video outlines cannot accurately extract and identify the video outlines. Summary of the invention

[0005] The present invention provides a method, device, equipment and medium for generating a video outline based on courseware recognition, so as to solve the technical problem that the existing solution for automatically identifying a video outline cannot accurately extract and identify the video outline.

[0006] In a first aspect, an embodiment of the present application provides a method for generating a video outline based on courseware recognition, the method for generating a video outline based on courseware recognition comprising:

[0007] Get the teaching courseware played during the classroom video recording and the page switching events of the teaching courseware;

[0008] According to the page switching event, confirm the displayed target courseware page and the display time period corresponding to each target courseware page;

[0009] The page content in each target courseware page is screened according to the preset content screening rules to obtain the target page content with the highest priority corresponding to each target courseware page;

[0010] The target page content corresponding to the target courseware page is used as the video outline information of the corresponding display period in the classroom video.

[0011] In a second aspect, an embodiment of the present application further provides a video outline generation device based on courseware recognition, the video outline generation device based on courseware recognition comprising:

[0012] A classroom data acquisition unit, used to acquire the teaching courseware played during the classroom video recording process and the page switching events of the teaching courseware;

[0013] A target page confirmation unit, used to confirm the displayed target courseware page and the display period corresponding to each target courseware page according to the page switching event;

[0014] The target content screening unit is used to screen the page content in each target courseware page according to the preset content screening rules to obtain the target page content with the highest priority corresponding to each target courseware page;

[0015] The outline information adding unit is used to use the target page content corresponding to the target courseware page as the video outline information of the corresponding display period in the classroom video.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, the electronic device comprising:

[0017] one or more processors;

[0018] a memory for storing one or more computer programs;

[0019] When one or more computer programs are executed by one or more processors, the electronic device implements the video outline generation method based on courseware recognition as in the first aspect.

[0020] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video outline generation method based on courseware recognition as in the first aspect.

[0021] The above-mentioned video outline generation method, device, equipment and medium based on courseware identification, in which the teaching courseware played during the classroom video recording and the page switching event of the teaching courseware are obtained; the displayed target courseware page and the display period corresponding to each target courseware page are confirmed according to the page switching event; the page content in each target courseware page is respectively screened according to the preset content screening rules to obtain the target page content with the highest priority corresponding to each target courseware page; the target page content corresponding to the target courseware page is used as the video outline information of the corresponding display period in the classroom video. By recording the data source and change event that causes the main display content in the classroom video to change, confirming the occurrence time of the change event and the core content of the page after the occurrence, and using the core content as the outline information of the corresponding page display period, without the need to identify the classroom video content, through the parallel recording of two-way related information, the processing requirement is simple. The accurate outline information of the information that requires complex processing to achieve content structure analysis can be obtained, and the accurate structural division and outline information identification of the classroom video based on the teaching content are realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A method flow chart of a method for generating a video outline based on courseware recognition provided in an embodiment of the present application.

[0023] Figure 2 This is a diagram of the screen structure of the classroom video.

[0024] Figure 3 A schematic diagram of the relationship between page switching events and the timeline in the video outline generation method based on courseware recognition provided in an embodiment of the present application.

[0025] Figure 4 A schematic diagram showing an exemplary distribution of the page content of a target courseware page in the video outline generation method based on courseware recognition provided in an embodiment of the present application.

[0026] Figure 5 A schematic diagram of the display of a video outline generated by the video outline generation method based on courseware recognition provided in an embodiment of the present application.

[0027] Figure 6 A schematic diagram of the structure of a video outline generation device based on courseware recognition provided in an embodiment of the present application.

[0028] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0030] It should be noted that due to space limitations, this application description does not list all optional implementation methods. After reading this application description, those skilled in the art should be able to understand that as long as the technical features do not contradict each other, any combination of technical features can constitute an optional implementation method.

[0031] The following is a detailed description of the various embodiments of the present invention.

[0032] With the development of information technology, the forms of computer-assisted teaching are constantly enriched and the effects are constantly improved. For example, an interactive tablet is configured in the teaching scene, and the electronic display and electronic recording of teaching information are realized based on the interactive tablet. The interactive tablet mentioned in this solution can be an integrated device that controls the content displayed on the display tablet and realizes human-computer interaction through touch technology. It integrates one or more functions such as a projector, an electronic whiteboard, a screen, a speaker, a television, and a video conferencing terminal.

[0033] Generally speaking, the interactive tablet includes at least one screen. For example, the interactive tablet is equipped with a screen with a touch function, and the screen can be a capacitive screen, a resistive screen, or an electromagnetic screen. In this embodiment, further, the user can implement touch operation by touching the screen with a finger or a stylus, and accordingly, the interactive tablet detects the touch position and responds according to the touch position to implement the touch function. Typically, the interactive tablet is installed with at least one operating system, wherein the operating system includes but is not limited to the Android system, the Linux system, and the Windows system.

[0034] In the teaching scenario, the interactive tablet is an interactive device mainly used to support the electronic display and electronic recording of teaching information, and the software therein is mainly used to realize the display and recording function requirements in the teaching scenario. For example, the interactive tablet can install at least one application with a document presentation function, wherein the application can be an application that comes with the operating system; at the same time, an application downloaded from a third-party device or server is also installed. Optionally, in addition to the basic function of displaying the edited content, the program with the document presentation function also has other editing functions in the presentation process, such as inserting tables, inserting pictures, inserting graphics, drawing tables, drawing graphics, etc., to realize the real-time display of user information input and input content during the information presentation process, thereby improving the information presentation and interaction effect during the teaching-related document presentation process. For example, the interactive tablet can install at least one application with a whiteboard function, and teachers can realize real-time writing, recording, modification and saving in the teaching scenario based on this application.

[0035] When the interactive tablet is used alone, it is mainly suitable for teachers to electronically display teaching information to students in the same teaching scene during offline teaching, such as teaching courseware, simulation animations, simulation software, etc.; and to electronically record the blackboard writing, notes, annotations, etc. generated by teachers during teaching activities.

[0036] On the basis of the interactive tablet in the teaching scene, a camera can be set up in the teaching scene to record the classroom video during the teaching process. The camera lens faces the side where the interactive tablet is located. When implementing specific teaching activities, the interactive tablet is displayed on the screen according to the teacher's operation. From the relative position of the relevant elements in the teaching process, the camera's shooting targets are mainly the teacher and the interactive tablet, and the wall where the interactive tablet is located is the teacher's lecture background. When the classroom video recorded by the camera is used later, the classroom video is only obtained by shooting the camera toward a shooting area. The classroom video is not structured when it is shot and generated. The direct use of the classroom video requires the playback progress to be adjusted to locate the possible target teaching moment. For example, students who do not participate in the classroom recording usually need to watch the video carefully to roughly understand the content of the classroom explanation; when students who participate in the classroom recording need to trace back some knowledge points that they did not understand or were interested in after class, they also need to constantly search for the location through the progress bar; and when the teacher who participated in the classroom recording wants to do teaching reflection after class, he needs to constantly jump back and forth through the video progress bar to retrieve and locate the specific teaching clip. In order to improve the efficiency of subsequent browsing of classroom videos, we can consider using the information summary of each segmented content in the segmented content structure as the outline information of the classroom video based on the segmented content structure of the substantial content recorded in the classroom video, so as to obtain the video outline of the classroom video. Based on the video outline, we can quickly confirm the general content of each part of the classroom video, and realize fast jump viewing of a certain part of the content based on the video outline. The video outline identified by the classroom video implemented by the automatic content recognition solution in the general video processing scenario usually cannot accurately identify the segmented content structure of the classroom video with reference to the teaching content.

[0037] In view of the above technical problems, the embodiment of the present application proposes a method for generating a video outline based on courseware identification, by recording the data source and change events that cause the main display content in the classroom video to change, confirming the time when the change event occurs and the core content of the page after the occurrence, and using the core content as the outline information of the corresponding page display period, without the need to identify the classroom video content, through the parallel recording of two channels of related information, the one channel of information with simple processing requirements can be processed to obtain accurate outline information of the one channel of information that requires complex processing to achieve content structure analysis, thereby achieving accurate structural division and outline information identification of classroom videos based on teaching content. The various embodiments of the present invention are described in detail below.

[0038] Figure 1This is a method flow chart of a method for generating a video outline based on courseware recognition provided in an embodiment of the present application. The method for generating a video outline based on courseware recognition is applied to an electronic device that processes relevant images of a teaching scene, and is implemented by an electronic device, such as a recording and broadcasting host, a cloud server that stores and manages teaching data, etc. Figure 1 As shown, the video outline generation method based on courseware recognition includes steps S110 to S140:

[0039] Step S110: Acquire the teaching courseware played during the classroom video recording and the page switching event of the teaching courseware.

[0040] Taking the implementation based on the recording and broadcasting host as an example, on the basis of the interactive tablet in the teaching scene, a camera and a recording and broadcasting host can be set up in the teaching scene, and both the camera and the interactive tablet are connected to the recording and broadcasting host. The lens of the camera faces the side where the interactive tablet is located. From the relative position of the relevant elements in the teaching process, the shooting targets of the camera when recording the classroom video include the teacher and the interactive tablet at a specific shooting moment. The interactive tablet is used to display the teaching courseware, that is, the teaching courseware is displayed in the recording area of ​​the classroom video. The picture captured by the camera is transmitted to the recording and broadcasting host in real time through the corresponding connection. While the interactive tablet displays on the screen according to the teacher's operation, the page switching event in operation is transmitted to the recording and broadcasting host in real time through the corresponding connection. The page switching event specifically refers to the event of operating the editing software of the teaching courseware to enter the slide show mode, the event of switching the courseware page in the slide show mode from one page to another, or the event of ending the slide show mode. The courseware page is displayed in full screen in the slide show mode.

[0041] In the process of teachers organizing teaching activities, there may be many operations that occur on the interactive tablet, such as annotation operations, blackboard operations, etc., where the file structure of the teaching courseware is usually based on pages as the basic unit. During the display of the teaching courseware, one page of content is displayed each time. The switching of the specific display content can be automatic switching, or the teacher can trigger the switching through laser pens, touch operations, mouse operations, keyboard shortcuts, etc. In the embodiment of the present application, all events that cause the pages of the teaching courseware to switch are defined as page switching events. Based on the above screen transmission and page switching events, the original data corresponding to the change in the interactive tablet display screen, that is, the page switching event, is received synchronously (or at a very small interval, which is equivalent to synchronization) from the interactive tablet in the screen received by the recording and broadcasting host from the camera. At this time, it can be regarded as realizing the recording of page switching events and classroom videos based on the same timeline.

[0042] like Figure 2As shown, when the teaching scene is a classroom 10, the classroom video is obtained by shooting the camera toward the teacher's lecture background. From the perspective of the structure of the screen content of the classroom video, it includes the foreground with the teacher 30 and the interactive tablet 20 as the main body, and the explanation background with the wall as the main body. The content in the classroom video is the video screen. Although the changes in the displayed content in the interactive tablet 20 and the movement of the teacher 30 may cause changes in the content details of the video screen, because the state of the camera remains basically unchanged, the layout of the classroom 10 also remains basically unchanged, and the structure of the screen content in the classroom video also remains basically unchanged. If the acquisition of the classroom video is completed by the recording and broadcasting host, the recording and broadcasting host can transmit data with the data processing related hardware for realizing the demonstration of teaching courseware through various existing connection methods (wired or wireless). For example, if the hardware device for realizing the data processing of the courseware demonstration is an interactive tablet, the recording and broadcasting host obtains the teaching courseware from the interactive tablet; if the hardware device for realizing the data processing of the courseware demonstration is a laptop connected to the interactive tablet, the recording and broadcasting host obtains the teaching courseware from the laptop. In addition, the hardware device for realizing the courseware demonstration may also be a projector, and the recording and broadcasting host obtains the teaching courseware from the computer host currently connected to the projector.

[0043] It should be understood that the recording and broadcasting host is only an optional implementation method. It can also be based on a cloud server. If the classroom video is processed in real time, the page switching event is also generated and received in real time, which is equivalent to recording the page switching event and the classroom video based on the same timeline through real-time processing. If the video outline is generated after the classroom video is recorded, considering that the starting time of the classroom video recording is not necessarily synchronized with the starting time of the teacher's operation of the teaching courseware, in order to ensure that the page switching event is aligned with the change of the interactive flat-panel display content in the classroom video, when recording the classroom video and recording the page switching event, in addition to generating a classroom video with the recording length as the timeline, it is also necessary to record the system time when the classroom video starts recording. Correspondingly, each page switching event also needs to record the system time, so as to realize the target page content in the target courseware page corresponding to the page switching event as the video outline information of the classroom video, and the final video outline information can accurately correspond to the changes in the actual screen in the classroom video because of the system time. When specifically realizing the acquisition and recording of basic data and information, each page switching event at least needs to record the occurrence time of the page switching event and the page information of the page displayed after switching. For the entire processing process of the embodiment of the present application, the outline information of the classroom video is generated with reference to the courseware page displayed after each page switching event. The page switching event and the courseware page displayed after the switching are directly obtained from the data source. Therefore, the location for displaying the teaching courseware does not necessarily need to be within the recording area of ​​the classroom video. Figure 2What is shown is only an exemplary shooting picture. If the camera performs target tracking, the display of the teaching courseware may no longer be captured. The page switching event of the teaching courseware can always be received, and this solution can be implemented accordingly.

[0044] In addition, the classroom video and teaching courseware can be obtained separately. For example, before or after obtaining the classroom video from the camera, the user can upload the teaching courseware through another electronic device, and the recording and broadcasting host obtains the teaching courseware accordingly. That is, the embodiment of the present application does not limit the acquisition of the classroom video and the teaching courseware to a certain time sequence or synchronization relationship. It is only necessary to have the classroom video and the corresponding teaching courseware before the specific data processing process of the video outline generation method in the embodiment of the present application is implemented.

[0045] If the acquisition of classroom video is completed by the recording and broadcasting host, the recording and broadcasting host can transmit data with the camera through an HDMI (High Definition Multimedia Interface) cable or other multimedia protocol connection cables, that is, the classroom video can be sent from the camera to the recording and broadcasting host through any of the various multimedia protocol connection cables.

[0046] Step S120: confirming the displayed target courseware pages and the display time period corresponding to each target courseware page according to the page switching event.

[0047] The picture in the classroom video is the video picture. An important picture content of the video picture is the display picture of the teaching courseware. Corresponding to the relationship between the picture content and the display picture of the video picture, the change process of the picture content in the video picture is strongly related to the operation of the teaching courseware in the specific teaching process, that is, the annotation operation, blackboard operation, page switching operation, etc. of the teaching courseware will cause the picture content in the recorded classroom video to change.

[0048] For teaching courseware, one courseware page is displayed each time, and each courseware page usually has a content core. The annotation and blackboard writing operations performed in the process of displaying a courseware page are generally centered on the content core of the courseware page. Therefore, although the annotation and blackboard writing operations may cause changes in the picture content in the classroom video, it is still considered that the change occurs within the display stage of a courseware page. Based on this, when generating the video outline of the classroom video, the display time period of each courseware page is used as a reference for dividing the segmented content structure of the video outline, and the information summary of the courseware page is used as the outline information of each segmented content structure.

[0049] The display change of the courseware page is determined by the page switching operation, such as page turning (including forward page turning and backward page turning) and page jump. It should be understood that starting to display the courseware page and ending to display the courseware page are also a kind of page switching operation. In the embodiment of the present application, based on the specific information of the corresponding record of each page switching event, it is possible to obtain which courseware pages enter the display state caused by the page switching operation of the teacher during the teaching process, and the display time period corresponding to each courseware page. In the embodiment of the present application, considering that not all courseware pages of a teaching courseware may necessarily enter the display state, or not all courseware pages are necessarily effectively displayed, the courseware page that is only effectively displayed is defined as the target courseware page, and the display time period corresponding to the target courseware page and each target courseware page is obtained according to the statistics of the page switching event. Effective display refers to the display whose display duration reaches the preset threshold value.

[0050] For example, when a certain teaching courseware is used in the teaching process, the page switching event of the teacher in the teaching process is monitored, and the specific moment when the page switching event occurs each time and the page information displayed after the page switching event are recorded. Of course, the recording method of the page switching event can also be a process of comprehensive information processing, such as recording the display period of the target courseware page that has recently ended display according to the page change and the specific moment, that is, recording the display start time and display end time of the target courseware page that has recently ended display. If the interval between two adjacent switching operations is very short (for example, 1 second or 2 seconds), it can be considered that the previous switching operation is only to realize the transition of the display of the target courseware page (for example, quickly turning from page 1 to page 3, page 2 is transition), at this time, this series of page switching operations can be corrected as a page switching event, and the courseware page whose final display duration reaches the preset threshold value is used as the target courseware page, and the display period of the target courseware page obtained by this processing actually displays multiple courseware pages, but only one courseware page is normally displayed, and the remaining courseware pages are transitional displays, without actual display significance, and there is no need to generate outline information accordingly.

[0051] The specific display time period confirmation method can be designed according to the time information recording method in the page switching event. For example, the moment when the classroom video starts recording is directly used as the revelation moment of the first segmented content structure, and the courseware page before the first page switching event is used as the first target courseware page. Then the first target courseware page may be the first courseware page used in the teaching activity (before starting to record the classroom video, the teacher has operated the teaching courseware to start the display of the courseware page, and the courseware page is displayed in full screen from the beginning of the classroom video). The first target courseware page may also be empty (the slide show mode is entered after the classroom video is recorded, and there is no courseware page displayed in full screen at the beginning of the classroom video). Or directly divide the display time period based on the existence of a clear target courseware page as the standard, that is, only when there is a clear target courseware page display, it is necessary to confirm the corresponding display time period, and only then is there a necessity to confirm the outline information in the corresponding segmented content structure in the classroom video. When the page switching event and the classroom video are recorded based on the same timeline, the accurate division of the display time period can be achieved at any time when the slide show mode is entered.

[0052] like Figure 3 As shown, in a certain teaching activity, the recording of the classroom video starts at time t0. At time t1, the teacher performs the slide show operation, and a page switching event occurs to enter the slide show mode. The first page of the 15 pages of the teaching courseware is displayed, and then the second page, the third page, and the fourth page are switched to the actual page at t2, t3, and t4 respectively. In this process, a classroom video starting from time t0 and page switching events corresponding to times t1, t2, t3, and t4 are generated.

[0053] Another exemplary specific implementation method is to assume that the slide show mode is entered before the recording of the classroom video begins. At 20 seconds after the recording of the classroom video begins, the teacher switches from page 1 to page 2. After 2 minutes of speaking, the teacher switches to page 5, and ends the course recording 10 minutes later. The recording format of each page switching event is (page start time (in seconds), page end time (in seconds), courseware page number), and the above page switching event can be recorded as: [(0,20,1), (20,140,2), (140,740,5)]; each page switching event is recorded as (page switching time (in seconds), page number before switching, page number after switching), and the above page switching event can be recorded as: [(20,1,2), (140,2,5), (740,5,0)]. For the former recording method, the display period can be directly confirmed from the first two digits of the recording format of the page switching event, and the target courseware page corresponding to the display period can be confirmed from the third digit. For the latter recording method, secondary processing is required to confirm the display period and the corresponding target courseware page. The specific processing result is based on the time of the page switching event and the target courseware pages before and after the switching, and the same confirmation result as the previous recording method can be obtained.

[0054] Step S130: performing content screening on the page content in each target courseware page according to a preset content screening rule to obtain the target page content with the highest priority corresponding to each target courseware page.

[0055] like Figure 4 As shown, in the target courseware page 201, multiple page contents may be included. For example, in a certain courseware page of a teaching courseware of "Knowing Graphics", an activity theme 202 for starting classroom activities after new knowledge is imparted is included. The activity theme mainly classifies multiple graphic elements 203, and different classification methods and classification results can be recorded in the input box 204. These page contents are not suitable to be used as the outline information of the display period corresponding to the target courseware page, so the page contents need to be screened according to the preset content screening rules to obtain the target page content with the highest priority corresponding to each target courseware page.

[0056] In the teaching courseware edited by the teaching courseware editing software, in order to highlight the key points of information transmission, each page content is recorded with corresponding various parameters, so as to highlight the key points and have clear levels when it is specifically displayed. In the process of specifically implementing the content screening of the target courseware page, the page content in each target courseware page is subjected to parameter extraction to obtain content attribute parameters; according to the priority and / or weight value of all the content attribute parameters of each page content, the content is screened according to the preset content screening rules to obtain the target page content with the highest priority corresponding to each target courseware page.

[0057] For the setting of specific content screening rules, for example, when the priorities of the attribute types of content attribute parameters are different, the higher the priority of the attribute type, the higher the priority of the corresponding page content; when the priorities of the attribute types of content attribute parameters are the same, the higher the sum of the weight values ​​of the content attribute parameters of the attribute types with the same priority, the higher the priority of the corresponding page content. Among them, the attribute types include classroom activities, ancient poems, mind maps, videos, Chinese character cards, and display attributes from high to low priority; display attributes include display position, font size, stroke thickness, and text length. If the teaching courseware is structured courseware, the content attribute parameters can be read directly from the structured information corresponding to each page content in each target courseware page.

[0058] For example, in some structured courseware, all courseware pages are usually stored in the courseware file package in XML format. The courseware page XML file is parsed to obtain content attribute parameters and corresponding page content. Use XML parsing tools to extract the specified content in the page content XML. For example, the xml.etree library in Python can be used to parse the courseware page XML file to extract the width and height attributes of the page, as well as the specified page content and content attribute parameters. An exemplary information that can be extracted is shown in Table 1.

[0059]

[0060]

[0061] Table 1

[0062] According to the page content and attributes in the courseware page, specific content is extracted as the target page content, and the target page content is used as the courseware page outline of the corresponding courseware page. The content of the courseware page is usually expressed with a rich type of elements, but when making courseware, the core information of the courseware page (outline, theme, key points, etc.) is usually also specially reflected in the courseware page, such as using special elements (such as classroom activities, mind maps, shapes, Chinese character cards, subject tools, etc.), or different font sizes, colors, thicknesses, different placements (usually placed in the center, upper middle, upper left corner of the courseware page), etc. Based on the above characteristics, the target page content of the courseware page can be roughly extracted based on the following rules: the courseware page contains page content with various attributes (special elements include: classroom activities, mind maps, ancient poems, videos, Chinese character cards), according to the characteristics of the subject, according to the specified priority (priority P0 is the highest, P4 is the lowest) and rules to confirm that one of the page contents is the target page content of a courseware page. In the case of containing special elements, the priority from high to low is classroom activities (the corresponding page content is the name of the classroom activity, for example Figure 3In the "Fun Classification" in the "List of Interesting Things", ancient poems (the corresponding page content is the title of the ancient poems), mind maps (the corresponding page content is the root node name of the mind map), videos (the corresponding page content is the video name), and Chinese character cards (the corresponding page content is a certain number of words). In the case of no special elements, the specific page content can be scored according to the preset weight value based on the display attributes, and the part of the page content with the highest weight value is used as the target page content. An exemplary way of scoring according to the display attributes is shown in Table 2.

[0063]

[0064]

[0065] Table 2

[0066] For example, in a certain courseware page, the width width = 1280, the height height = 780, and in a text box in the courseware page, the text content text = "Import new lesson", the font is 36, bold, and the coordinates in the page are (600, 20). According to the above rules, the score calculation is shown in Table 3.

[0067]

[0068] Table 3

[0069] The outline likelihood score of the text is: 0.531×0.01+0.974×0.3+0.120×0.05+1×0.3+1×0.01-0×0.1=0.61351.

[0070] Based on the above calculation method, the page content with the highest score ranking in the courseware page is finally taken as the target page content of the courseware page. In particular, since some courseware pages may only present long text segments, when the length of the first text element exceeds 20, the first 20 characters are intercepted as the target page content. It should be noted that the above exemplary content screening rules are only designed for certain page contents of a certain subject, and do not represent the only implementation method. More content screening rule details can be provided for more subjects and more content attribute parameters.

[0071] Step S140: Using the target page content corresponding to the target courseware page as the video outline information of the corresponding display period in the classroom video.

[0072] After extracting the target page content in step S110-step S130, it is equivalent to obtaining the video outline information of the display period corresponding to each target courseware page used in the classroom video recorded in a class. Assume that the target courseware page used by the teacher for teaching is X = (x1, x2, x5), xi is the i-th page of the courseware. By extracting the target page content described above, the corresponding target page content is extracted page by page to obtain the target page content sequence, that is, the video outline information sequence C, C = (c1, c2, c5), where c i It is the video outline information of the display period corresponding to the i-th courseware page in the classroom video.

[0073] like Figure 5 As shown, the video outline information 303 and the classroom video 302 are displayed in the same video playback window 301. The video outline information 302 can display the target page content corresponding to each target courseware page as the outline information of the corresponding content segment structure, and correspondingly present the time start and end information of the display period (including the start time and the end time). When a trigger operation for a certain outline information is received in the video outline information 303, the classroom video 302 jumps to the start time of the corresponding display period.

[0074] The above-mentioned video outline generation method based on courseware identification obtains the teaching courseware played during the classroom video recording process and the page switching event of the teaching courseware; confirms the displayed target courseware page and the display time period corresponding to each target courseware page according to the page switching event; performs content screening on the page content in each target courseware page according to the preset content screening rules, and obtains the target page content with the highest priority corresponding to each target courseware page; and uses the target page content corresponding to the target courseware page as the video outline information of the corresponding display time period in the classroom video. By recording the data source and change events that cause the main display content in the classroom video to change, confirming the time when the change event occurs and the core content of the page after the change event occurs, and using the core content as the outline information of the corresponding page display time period, without the need to identify the classroom video content, through the parallel recording of two channels of related information, the processing of one channel of information with simple processing requirements can be processed, and the accurate outline information of one channel of information that requires complex processing to achieve content structure analysis can be obtained, thereby realizing the accurate structural division and outline information identification of the classroom video based on the teaching content.

[0075] Figure 6 The structure diagram of a video outline generation device based on courseware recognition provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the video outline generation device based on courseware recognition includes a classroom data acquisition unit 210, a target page confirmation unit 220, a target content screening unit 230 and an outline information adding unit 240.

[0076] Among them, the classroom data acquisition unit 210 is used to obtain the teaching courseware played during the classroom video recording and the page switching events of the teaching courseware; the target page confirmation unit 220 is used to confirm the displayed target courseware page and the display time period corresponding to each target courseware page according to the page switching event; the target content screening unit 230 is used to screen the page content in each target courseware page according to the preset content screening rules, and obtain the target page content with the highest priority corresponding to each target courseware page; the outline information adding unit 240 is used to use the target page content corresponding to the target courseware page as the video outline information of the corresponding display time period in the classroom video.

[0077] Based on the above embodiment, the target content screening unit 230 includes:

[0078] A parameter extraction module is used to extract parameters from the page content in each target courseware page to obtain content attribute parameters;

[0079] According to the priority and / or weight value of all content attribute parameters of each page content, content screening is performed according to preset content screening rules to obtain the target page content with the highest priority corresponding to each target courseware page.

[0080] Based on the above embodiment, the content screening rules include:

[0081] In the case where the priorities of the attribute types of the content attribute parameters are different, the higher the priority of the attribute type, the higher the priority of the corresponding page content;

[0082] In the case where the priorities of the attribute types of the content attribute parameters are the same, the higher the sum of the weight values ​​of the content attribute parameters of the attribute types with the same priority is, the higher the priority of the corresponding page content is.

[0083] Based on the above embodiment, the teaching courseware is structured courseware;

[0084] Correspondingly, the parameter extraction module includes:

[0085] The structured information extraction module is used to read content attribute parameters from the structured information corresponding to each page content in each target courseware page.

[0086] Based on the above embodiment, the attribute types include classroom activities, ancient poems, mind maps, videos, Chinese character cards and display attributes in descending order of priority.

[0087] Based on the above embodiment, the display attributes include display position, font size, stroke thickness and text length.

[0088] Based on the above embodiment, the page switching event and the classroom video are recorded based on the same timeline.

[0089] The video outline generation device based on courseware recognition provided in the embodiment of the present application is included in an electronic device and can be used to execute the corresponding video outline generation method based on courseware recognition provided in the above embodiment, and has corresponding functions and beneficial effects.

[0090] It is worth noting that in the above-mentioned embodiment of the video outline generation device based on courseware recognition, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0091] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device includes a processor 310 and a memory 320, and may also include an input device 330, an output device 340 and a communication device 350; the number of the processor 310 in the electronic device may be one or more, Figure 7 A processor 310 is taken as an example; the processor 310, the memory 320, the input device 330, the output device 340 and the communication device 350 in the electronic device can be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0092] The memory 320, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the video outline generation method based on courseware recognition in the embodiment of the present application. The processor 310 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 320, that is, realizing the above-mentioned video outline generation method based on courseware recognition.

[0093] The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 320 may further include a memory remotely arranged relative to the processor 310, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0094] The input device 330 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 340 may include a display device such as a display screen.

[0095] The above-mentioned electronic device includes a video outline generation device based on courseware recognition, which can be used to execute any video outline generation method based on courseware recognition and has corresponding functions and beneficial effects.

[0096] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it is used to perform relevant operations in the video outline generation method based on courseware recognition provided in any embodiment of the present application, and has corresponding functions and beneficial effects.

[0097] Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products.

[0098] Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0099] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0100] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0101] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0102] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A video outline generation method based on courseware recognition, characterized in that: include: Obtain the teaching courseware played during the classroom video recording and the page switching event of the teaching courseware; Confirming the displayed target courseware page and the display time period corresponding to each target courseware page according to the page switching event; Performing content screening on the page content of each target courseware page according to a preset content screening rule to obtain the target page content with the highest priority corresponding to each target courseware page; The target page content corresponding to the target courseware page is used as the video outline information of the corresponding display time period in the classroom video.

2. The video outline generation method based on courseware recognition according to claim 1 is characterized in that: The page content of each target courseware page is screened according to a preset content screening rule to obtain the target page content with the highest priority corresponding to each target courseware page, including: Extracting parameters from the page content in each target courseware page to obtain content attribute parameters; According to the priority and / or weight value of all content attribute parameters of each of the page contents, content screening is performed according to preset content screening rules to obtain the target page content with the highest priority corresponding to each of the target courseware pages.

3. The video outline generation method based on courseware recognition according to claim 2 is characterized in that: The content screening rules include: In the case where the priorities of the attribute types of the content attribute parameters are different, the higher the priority of the attribute type, the higher the priority of the corresponding page content; In the case where the priorities of the attribute types of the content attribute parameters are the same, the higher the sum of the weight values ​​of the content attribute parameters of the attribute types with the same priority is, the higher the priority of the corresponding page content is.

4. The video outline generation method based on courseware recognition according to claim 3 is characterized in that: The teaching courseware is structured courseware; Correspondingly, the page content in each target courseware page is subjected to parameter extraction to obtain content attribute parameters, including: Content attribute parameters are read from the structured information corresponding to each of the page contents in each of the target courseware pages.

5. The video outline generation method based on courseware recognition according to claim 4 is characterized in that: The attribute types include classroom activities, ancient poems, mind maps, videos, Chinese character cards, and display attributes in descending order of priority.

6. The video outline generation method based on courseware recognition according to claim 4 is characterized in that: The display attributes include display position, font size, stroke thickness and text length.

7. The method for generating a video outline based on courseware recognition according to any one of claims 1 to 6, characterized in that: The page switching event is recorded based on the same timeline as the classroom video.

8. A video outline generation device based on courseware recognition, characterized in that: include: A classroom data acquisition unit, used to acquire the teaching courseware played during the classroom video recording process and the page switching event of the teaching courseware; A target page confirmation unit, used to confirm the displayed target courseware page and the display time period corresponding to each target courseware page according to the page switching event; A target content screening unit, used to screen the page content in each of the target courseware pages according to a preset content screening rule, to obtain the target page content with the highest priority corresponding to each of the target courseware pages; The outline information adding unit is used to use the target page content corresponding to the target courseware page as the video outline information of the corresponding display period in the classroom video.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more computer programs; When the one or more computer programs are executed by the one or more processors, the electronic device implements the video outline generation method based on courseware recognition as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for generating a video outline based on courseware recognition as described in any one of claims 1 to 7 is implemented.

Images