A classroom teaching live broadcast recording method and device

Abstract

The present invention provides a classroom teaching live broadcast recording method and device, which relates to the field of live broadcast recording communication technology, wherein the method includes: when the first user starts the teaching live broadcast in the classroom, recording the live video until the live broadcast ends; pre-processing the live video to obtain the recorded teaching course; recording and publishing the recorded teaching course. The classroom teaching live broadcast recording method and device of the present invention, when the teaching staff starts the teaching live broadcast in the classroom, records the live video until the live broadcast ends, then automatically pre-processes the live video, identifies the segments to be removed in the live video and removes them, then encapsulates the course, and then records and publishes the recorded teaching course obtained by the pre-processing, which does not need to be completed by the teaching staff themselves, reduces the labor cost, improves the publishing speed of the recorded teaching course, avoids affecting the learning progress of the students, and improves the learning efficiency and learning experience of the students.

Description

Technical Field

[0001] The present invention relates to the technical field of live broadcast and recording communication, and in particular to a method and device for live broadcast and recording of classroom teaching. Background Art

[0002] At present, in order to improve the convenience of students' learning, many teaching staff choose to conduct live teaching in the classroom, and hope to record the video of the live broadcast so as to convert it into a recorded course for more students to learn or for on-site students to review.

[0003] However, due to the strong on-site nature of live teaching, recorded live videos often contain a lot of content that is not suitable for inclusion in recorded teaching courses, such as video clips of interactions with students, video clips of temporary interruptions in the teaching process, etc. Therefore, it is necessary to pre-process the recorded live videos.

[0004] However, this process is usually completed by teaching staff themselves, which requires a large investment of manpower and may delay the release of recorded teaching courses, affecting students' learning progress. Some teaching staff choose not to perform preprocessing, which can speed up the release, but may also lead to a decrease in students' learning efficiency and learning experience.

[0005] Based on this, a solution is urgently needed. Summary of the invention

[0006] One of the purposes of the present invention is to provide a method for recording live broadcast of classroom teaching. When the teaching staff starts live teaching in the classroom, the live video is recorded until the live broadcast ends, and then the live video is automatically pre-processed, the segments to be removed in the live video are identified and removed, and then the course is packaged. Then the recorded teaching course obtained by pre-processing is recorded and published, which does not need to be completed by the teaching staff themselves, reducing labor costs, improving the publishing speed of recorded teaching courses, avoiding affecting the learning progress of students, and improving the learning efficiency and learning experience of students.

[0007] An embodiment of the present invention provides a classroom teaching live broadcast and recording method, comprising:

[0008] When the first user starts a live teaching session in the classroom, the live video is recorded until the end of the live session;

[0009] Pre-process the live video to obtain recorded teaching lessons;

[0010] Record and broadcast teaching courses.

[0011] Optionally, preprocessing the live video to obtain a recorded teaching course includes:

[0012] Identify segments to be removed from live video;

[0013] Remove the segments that need to be removed from the live video;

[0014] The live video is packaged into a course after the segments to be removed to obtain the recorded teaching course.

[0015] Optionally, the identifying the segments to be removed from the live video includes:

[0016] Identify unvoiced segments from live video;

[0017] If the non-voice segment meets the first active silence condition, analyzing the start and end time periods of the non-voice segment;

[0018] Obtaining a movement trajectory map of the first user in the classroom during the start and end time periods;

[0019] If the moving trajectory diagram meets the second active silent condition, the segment without voice is regarded as a segment to be removed;

[0020] Among them, the first active silent condition includes:

[0021] The duration of the segment without vocalization exceeds the first duration threshold;

[0022] Moreover, the vocal semantic sets in the adjacent vocal segments before and after the non-vocal segments in the live video do not match multiple standard vocal semantic sets;

[0023] Moreover, the action sets of people in the segments without voice and the adjacent segments with voice before and after the segments without voice in the live video do not match the multiple standard action sets of people;

[0024] Among them, the second active silent condition includes:

[0025] The ratio of the target local trajectory segment to the total movement trajectory on the movement trajectory diagram exceeds the ratio threshold; wherein the relative position relationship between each trajectory point on the target local trajectory segment and the live broadcast camera position matches the standard relative position relationship.

[0026] Optionally, after the recorded teaching course is recorded and released, the method further includes:

[0027] When the second user orders the recorded teaching course, the recorded teaching course is interactively played to the second user.

[0028] Optionally, the interactively playing the recorded teaching course to the second user includes:

[0029] Displaying the recorded teaching course to the second user, and controlling the displayed recorded teaching course to start playing;

[0030] When the second user views the displayed recorded teaching course and the target content at the same time, if the target content meets the content linkage triggering condition, based on the first remaining playback time axis of the recorded teaching course and the second remaining playback time axis of the target content, the content linkage resources and the corresponding resource arrangement time period on the first remaining playback time axis are determined;

[0031] Arrange content linkage resources within the resource arrangement time period;

[0032] Based on the first remaining playback time axis after the content linkage resource arrangement is completed within the resource arrangement time period, controlling the displayed recorded teaching course to continue to be played;

[0033] The content linkage trigger conditions include:

[0034] The cumulative time duration during which the target content is viewed by the second user and is not moved by the second user exceeds a second time duration threshold;

[0035] Also, the content type of the target content matches the standard content type.

[0036] Optionally, determining the content linkage resources and the corresponding resource arrangement time period on the first remaining playback time axis based on the first remaining playback time axis of the recorded teaching course and the second remaining playback time axis of the target content includes:

[0037] Based on the content pairing constraint, the first remaining playback time axis and the second remaining playback time axis are content-paired to obtain a paired first content item and a second content item; wherein the first content item is from a first playback time period on the first remaining playback time axis, and the second content item is from a second playback time period on the second remaining playback time axis;

[0038] Extracting features from the first content item and the second content item to obtain a content feature set;

[0039] Generate resource screening conditions based on content feature sets;

[0040] Based on the resource screening conditions, content linkage resources are screened out from the content linkage resource library;

[0041] Using the overlapping time period between the first play time period and the second play time period as the resource arrangement time period;

[0042] Among them, content matching constraints include:

[0043] The association relationship between the first content item and the second content item matches the standard association relationship;

[0044] Furthermore, a start time difference and an end time difference between the first playback time period and the second playback time period do not exceed a time difference threshold.

[0045] An embodiment of the present invention provides a classroom teaching live broadcast and recording device, comprising:

[0046] A live broadcast recording module, used for recording the live broadcast video until the end of the live broadcast when the first user starts the live teaching in the classroom;

[0047] The video preprocessing module is used to preprocess the live video to obtain the recorded teaching course;

[0048] The recording and broadcasting publishing module is used to publish the recording and broadcasting teaching courses.

[0049] Optionally, the video preprocessing module preprocesses the live video to obtain a recorded teaching course, including:

[0050] Identify segments to be removed from live video;

[0051] Remove the segments that need to be removed from the live video;

[0052] The live video is packaged into a course after the segments to be removed to obtain the recorded teaching course.

[0053] Optionally, the video preprocessing module identifies segments to be removed from the live video, including:

[0054] Identify unvoiced segments from live video;

[0055] If the non-voice segment meets the first active silence condition, analyzing the start and end time periods of the non-voice segment;

[0056] Obtaining a movement trajectory map of the first user in the classroom during the start and end time periods;

[0057] If the moving trajectory diagram meets the second active silent condition, the segment without voice is regarded as a segment to be removed;

[0058] Among them, the first active silent condition includes:

[0059] The duration of the segment without vocalization exceeds the first duration threshold;

[0060] Moreover, the vocal semantic sets in the adjacent vocal segments before and after the non-vocal segments in the live video do not match multiple standard vocal semantic sets;

[0061] Moreover, the action sets of people in the segments without voice and the adjacent segments with voice before and after the segments without voice in the live video do not match the multiple standard action sets of people;

[0062] Among them, the second active silent condition includes:

[0063] The ratio of the target local trajectory segment to the total movement trajectory on the movement trajectory diagram exceeds the ratio threshold; wherein the relative position relationship between each trajectory point on the target local trajectory segment and the live broadcast camera position matches the standard relative position relationship.

[0064] Optionally, after the recording and broadcasting publishing module records and broadcasts the recorded and broadcasted teaching course, it further includes:

[0065] The interactive playback module is used to interactively play the recorded teaching course to the second user when the second user requests the recorded teaching course.

[0066] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0067] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0069] Figure 1 A schematic diagram of a classroom teaching live broadcast and recording method according to an embodiment of the present invention;

[0070] Figure 2 It is a schematic diagram of a classroom teaching live broadcast and recording device in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0072] The embodiment of the present invention provides a classroom teaching live broadcast and recording method, such as Figure 1 As shown, including:

[0073] S1. When the first user starts a live teaching session in the classroom, the live video is recorded until the end of the live broadcast;

[0074] In S1, the first user is a teaching staff; the first user enters the classroom, starts a live teaching session, and records the live video of the live session until the end of the live session;

[0075] S2, pre-processing the live video to obtain the recorded teaching course;

[0076] S3, recording and publishing the recorded teaching course;

[0077] In S3, recording and broadcasting release refers to publishing the recorded teaching course online, such as publishing it on an online course platform. After the recording and broadcasting is released, students who need to learn can watch the recorded teaching course online on demand.

[0078] Wherein, the S2, pre-processing the live video to obtain the recorded teaching course, includes:

[0079] S21, identifying segments to be removed from the live video;

[0080] In S21, the segments to be removed are video segments that are not suitable for inclusion in the recorded teaching course, such as video segments related to the barrage interaction with students, video segments related to the temporary interruption process in the teaching process, etc.; the current teaching process type can be identified during the live broadcast of the first user, and the relevant recorded video segments can be marked based on the teaching process type, and finally the segments to be removed can be identified based on the marks, for example: the first user interacts with the students during the live broadcast, and the teaching process type is barrage interaction, then the video segments recorded during the barrage interaction between the first user and the students are marked as barrage interaction, and finally all the video segments marked as barrage interaction are found as the segments to be removed;

[0081] S22, removing the segments to be removed from the live video;

[0082] In S22, the segment to be eliminated is identified and deleted from the live video;

[0083] S23, encapsulating the live video after the segments to be removed to obtain a recorded teaching course.

[0084] In S23, course encapsulation refers to converting the live video into an online course format after removing the segments that need to be removed, for example: adding course introductions such as course topic, course type, and lecturer.

[0085] When the teaching staff starts a live teaching in the classroom, this application records the live video until the end of the live broadcast, and then automatically pre-processes the live video, identifies and removes the segments that need to be removed in the live video, and then packages the course. The recorded teaching course obtained by pre-processing is then recorded and released. The teaching staff does not need to complete it by themselves, which reduces labor costs, increases the speed of releasing recorded teaching courses, avoids affecting the students' learning progress, and improves the students' learning efficiency and learning experience.

[0086] In one embodiment, the step S21, identifying segments to be removed from the live video, includes:

[0087] S211, determining a segment without human voice from the live video;

[0088] In S211, the silent segment refers to a video segment in which there is no continuous human sound. Since only the first user speaks during the live broadcast, that is, the video segment in which the first user does not speak continuously; when the first user is teaching live, there are two situations in which he does not speak continuously. The first situation is active silence, which means that the first user actively chooses silence and subjectively does not want the silent segment of the corresponding part to be included in the recorded teaching class. For example, the first user comes down from the relatively cramped podium in the classroom and walks into other areas with larger space to demonstrate the action. After the demonstration, he walks to the podium from the other area with his back to the live broadcast camera and without speaking, and prepares to act on the blackboard. , then the process of walking from the other area to the podium with the back to the live broadcast camera and without speaking is the first user's active silence; the second is passive silence, which means that the first user passively chooses silence and subjectively hopes that the corresponding part of the unvoiced segment will be included in the recorded teaching class. For example: the first user chooses to let the students studying in the live broadcast room take a break and actively gives them a 10-minute break to adjust their learning status. Students who order recorded classes also need to rest to adjust their status. Therefore, the recorded teaching class should include this part of the unvoiced segment; the next step is to determine whether the unvoiced segment is generated by the first user in the case of active silence. If so, it should be used as a segment to be eliminated;

[0089] S212: if the non-voice segment meets the first active silence condition, analyzing the start and end time periods of the non-voice segment;

[0090] In S212, when the segment without voice meets the first active silence condition, it is preliminarily determined that the segment without voice is generated by the first user in the active silence condition, and then subsequent operations are performed; the start and end time periods are the time periods for recording the segment without voice;

[0091] S213, obtaining a movement trajectory diagram of the first user in the classroom during the start and end time periods;

[0092] In S213, the movement trajectory graph includes a movement trajectory generated by the first user moving in the classroom during the start and end time periods. The movement trajectory can be drawn based on the dynamic change of the position of the first user in the classroom determined based on the recorded video. The movement trajectory graph also includes a live broadcast camera position, which is the position of a camera device used by the first user to perform live teaching, such as a position of a smart phone.

[0093] S214, if the moving trajectory diagram meets the second active silent condition, the segment without voice is regarded as a segment to be removed;

[0094] When the movement trajectory graph meets the second active silence condition, it means that it is completely determined that the segment without voice is generated by the first user in the active silence condition, and the segment without voice is regarded as a segment to be removed and waits for removal;

[0095] Among them, the first active silent condition includes:

[0096] Condition 1: The duration of the non-voice segment does not exceed the first duration threshold;

[0097] Moreover, in condition 2, the vocal semantic sets of the vocal segments before and after the vocal segments without vocals in the live video do not match multiple standard vocal semantic sets;

[0098] Moreover, in condition three, the action sets of people in the segments without voice and the adjacent segments with voice before and after the segments without voice in the live video do not match the multiple standard action sets of people;

[0099] Among them, the second active silent condition includes:

[0100] Condition 4: The ratio of the target local trajectory segment to the total movement trajectory on the movement trajectory diagram exceeds the ratio threshold; wherein the relative position relationship between each trajectory point on the target local trajectory segment and the live broadcast camera position matches the standard relative position relationship.

[0101] In condition one, the first duration threshold can be 5 seconds; the duration of the first user's active silence will not be very short, so condition one is set; in condition two, the standard voice semantic set includes multiple semantics that jointly reflect the first user's passive silence, such as: "the following time", "half-time", "rest for 10 minutes" and "adjust learning status"; the voice semantic set in the voice segments adjacent to the unvoiced segment includes the semantics of multiple speech contents in the voice segment; only when the voice semantic set in the voice segments adjacent to the unvoiced segment in the live video does not match multiple standard voice semantic sets can it be preliminarily guaranteed. The segment without voice is generated by the first user in active silence; in condition three, the standard personnel action set includes multiple actions that jointly reflect the situation where the first user generates passive silence, such as teaching actions, gestures to instruct students to take a break, etc.; the personnel action set in the segment without voice and the human voice segments adjacent to the segment without voice includes the segment without voice and multiple actions generated by the first user in the segment without voice; only when the personnel action set in the segment without voice and the human voice segments adjacent to the segment without voice in the live video do not match multiple standard personnel action sets, can it be preliminarily guaranteed that the segment without voice is generated by the first user in active silence. In summary, conditions one, two and three are set in the first active silence condition. When the segment without voice meets conditions one, two and three at the same time, it can be preliminarily determined that the segment without voice is generated by the first user in active silence.

[0102] In condition 4, the ratio threshold can be 0.65; the standard relative position relationship is the position relationship between the first user and the live broadcast camera when the user is actively silent, for example: the first user moves with his back to the live broadcast camera, that is, the moving direction of the trajectory point (in the moving trajectory, the moving direction of a certain trajectory point is the direction of the line connecting the trajectory point to the next trajectory point) is the direction of the shooting with his back to the live broadcast camera; when the ratio of the target local trajectory segment to the total moving trajectory exceeds the ratio threshold, it means that the first user has more cases of active silence, and it is completely determined that the segment without voice is generated by the first user in the case of active silence. Therefore, condition 4 is set in the second active silence condition. When the moving trajectory graph meets condition 4, it can be determined that the segment without voice is generated by the first user in the case of active silence.

[0103] When identifying segments to be removed from a live video, the embodiment of the present invention identifies segments to be removed that are generated by the first user under special circumstances, that is, active silence, which greatly improves the applicability of the system; when the segment without voice meets the first active silence condition, it is preliminarily determined that the segment without voice is generated by the first user under active silence, and then subsequent operations are performed at this time to accurately determine the timing of subsequent operations, thereby reducing the system's recognition resource usage and improving recognition efficiency; the first active silence condition and the second active silence condition are introduced to gradually determine in levels whether the segment without voice is generated by the first user under active silence, which greatly improves the comprehensiveness and accuracy of recognition.

[0104] In one embodiment, after S3, recording and broadcasting the recorded teaching course, further includes:

[0105] S4. When the second user requests the recorded teaching course, the recorded teaching course is interactively played to the second user;

[0106] In S4, the second user is a student taking an online course; the second user can order a recorded teaching course, and when the second user orders the recorded teaching course, the recorded teaching course is interactively played to the second user;

[0107] Wherein, in S4, interactively playing the recorded teaching course to the second user includes:

[0108] S41, displaying the recorded teaching course to the second user, and controlling the displayed recorded teaching course to start playing;

[0109] In S41, during interactive playback, the recorded teaching course is first displayed to the second user, and the user is controlled to start playing; the second user can view the displayed recorded teaching course that starts playing;

[0110] S42, when the second user views the displayed recorded teaching course and the target content at the same time, if the target content meets the content linkage triggering condition, based on the first remaining playback time axis of the recorded teaching course and the second remaining playback time axis of the target content, determine the content linkage resources and the corresponding resource arrangement time period on the first remaining playback time axis;

[0111] In S42, while viewing the displayed recorded teaching course, the second user will also view the target content, for example: related reference materials that can be automatically played page by page, related exercises that can be played page by page, etc.; the interaction in the embodiment of the present invention refers to content linkage between the recorded teaching course and the target content; if the target content meets the content linkage triggering condition, it is determined that the recorded teaching course can be content-linked with the target content, and then subsequent operations are performed; the first remaining playback time axis of the recorded teaching course has multiple time nodes for the future playback of the recorded teaching course and the corresponding playback content, and correspondingly, the second remaining playback time axis of the target content has multiple time nodes for the future playback of the target content and the corresponding playback content; content linkage resources refer to the material resources that need to be arranged in the recorded teaching course to enable the recorded teaching course to be content-linked with the target content, and the resource arrangement time period is the time period in which the content linkage resource arrangement can be performed;

[0112] S43, arranging content linkage resources within a resource arrangement time period;

[0113] In S43, the content linkage resource is then arranged within the resource arrangement time period; when the content linkage resource is arranged within the resource arrangement time period, the content linkage resource is controlled to be continuously displayed within the resource arrangement time period and to be operated by the second user;

[0114] S44, based on the first remaining playback time axis after the content linkage resource arrangement is completed within the resource arrangement time period, controlling the displayed recorded teaching course to continue to be played;

[0115] In S44, after the content linkage resource arrangement is completed, based on the first remaining playback time axis after the content linkage resource arrangement is completed, the displayed recorded teaching course is controlled to continue to be played, that is, when the time for playing the recorded teaching course in the future enters the resource arrangement time period, the content linkage resource is also played to achieve content linkage between the recorded teaching course and the target content;

[0116] The content linkage trigger conditions include:

[0117] Condition 5: the cumulative time for which the target content is viewed by the second user and not moved by the second user exceeds a second time threshold;

[0118] Furthermore, in condition six, the content type of the target content matches the standard content type.

[0119] In condition five, being moved by the second user refers to movement such as dragging, and the second duration threshold can be 20 seconds; in condition six, the standard content type is a content type that represents the target content that is suitable for content linkage, such as: related reference materials that can be automatically played page by page, and related exercises that can be played page by page; when it is necessary to link the recorded teaching class with the target content, the second user needs to keep paying attention to the target content, so condition five is set; in addition, the second user also needs the content type of the target content to be suitable for content linkage, so condition six is ​​set. In summary, by setting conditions five and six in the content linkage trigger conditions, if the target content meets the content linkage trigger conditions, it is determined that the recorded teaching class can be linked with the target content.

[0120] The embodiment of the present invention links the recorded teaching course with the target content to realize interactive playback of the recorded teaching course to the second user, which greatly improves the user experience of viewing the recorded teaching course and is more intelligent. A content linkage trigger condition is set. If the target content meets the content linkage trigger condition, it is determined that the recorded teaching course can be linked with the target content. At this time, subsequent operations are performed to accurately determine the linkage timing, reduce the interactive resource occupation of the system, and improve the interactive efficiency of the system. Based on the first remaining playback time axis of the recorded teaching course and the second remaining playback time axis of the target content, the content linkage resources and the corresponding resource layout time period on the first remaining playback time axis are determined, and it is efficiently determined which resource to use in which time period to link the recorded teaching course with the target content, thereby further improving the interactive efficiency of the system.

[0121] In one embodiment, in S42, based on the first remaining playback time axis of the recorded teaching course and the second remaining playback time axis of the target content, determining the content linkage resources and the corresponding resource arrangement time period on the first remaining playback time axis includes:

[0122] S421, based on the content pairing constraint, perform content pairing on the first remaining playback time axis and the second remaining playback time axis to obtain a paired first content item and a paired second content item; wherein the first content item is from a first playback time period on the first remaining playback time axis, and the second content item is from a second playback time period on the second remaining playback time axis;

[0123] In S421, after pairing, the first content item is the playback content of the recorded teaching course in the first playback time period, and correspondingly, the second content item is the playback content of the target content in the second playback time period;

[0124] S422, extracting features from the first content item and the second content item to obtain a content feature set;

[0125] In S422, the content feature set includes the play content features of the first content item and the second content item, for example, the content type of the first content item and the content type of the second content item, etc.;

[0126] S423, generating resource screening conditions based on the content feature set;

[0127] In S423, the resource screening condition generated is a condition for screening out content linkage resources suitable for content linkage between the first content item and the second content item. For example, if the play content feature in the content feature set is that the content type of the first content item is the definition of L'Hôpital's rule in advanced mathematics and the content type of the second content item is a test question related to the definition of L'Hôpital's rule, then the resource screening condition generated is a condition that enables the definition of L'Hôpital's rule to be content-linked with the test question related to the definition of L'Hôpital's rule;

[0128] S424, based on the resource screening condition, screening content linkage resources from the content linkage resource library;

[0129] In S424, there are a large number of content linkage resources in the content linkage resource library, and content linkage resources are selected from them based on the resource screening condition. For example, if the resource screening condition is to enable content linkage between the definition of L'Hôpital's rule and test questions related to the definition of L'Hôpital's rule, the selected content linkage resources may be an information box containing prompt information for prompting the second user to complete test questions related to the definition of L'Hôpital's rule, an information box containing extended test questions related to the definition of L'Hôpital's rule other than the second content item, etc.;

[0130] S425, using the overlapping time period between the first play time period and the second play time period as a resource arrangement time period;

[0131] In S425, only when the second user views the first content item and the second content item together is the optimal output time for the content linkage resource, and therefore, the overlapping time period between the first playback time period and the second playback time period is used as the resource arrangement time period;

[0132] Among them, content matching constraints include:

[0133] Constraint 1: the association relationship between the first content item and the second content item matches the standard association relationship;

[0134] Furthermore, constraint 2: the start time difference and the end time difference between the first playback time period and the second playback time period do not exceed the time difference threshold.

[0135] In constraint one, the standard association relationship refers to the association relationship that represents the content linkage between the first content item and the second content item, such as: adapted teaching content and exercises, theoretical knowledge and practical cases, and basic knowledge and advanced content, etc.; the first content item and the second content item are constrained to comply with constraint one, so that the paired first content item and the second content item can be content-linked. In constraint two, the start time difference between the first play time period and the second play time period refers to the absolute value of the difference between the start times of the two time periods, and correspondingly, the end time difference between the first play time period and the second play time period refers to the absolute value of the difference between the end times of the two time periods; the time difference threshold can be 8 seconds; ensuring that the start time difference and the end time difference do not exceed the time difference threshold can make the first content item and the second content item have a small difference in the play time stage, so that actual content linkage operation can be performed. In summary, setting constraint one and constraint two in the content pairing constraint ensures that the paired first content item meets both constraint one and constraint two, and fully ensures that the paired first content item and the second content item can be content-linked.

[0136] The embodiment of the present invention introduces a content pairing constraint. Under the constraint of the content pairing constraint, it is ensured that the first remaining playback time axis and the second remaining playback time axis are accurately paired with each other, and the first content item and the second content item that can be content-linked are determined, which greatly improves the work efficiency of the system; introduces a content feature set, and generates resource screening conditions based on the content feature set to screen out suitable content linkage resources from the content linkage resource library, which further improves the work efficiency of the system and the accuracy of content linkage resource determination; uses the overlapping time period between the first playback time period and the second playback time period as a resource arrangement time period, which greatly improves the suitability of the resource arrangement time period setting.

[0137] The embodiment of the present invention provides a classroom teaching live broadcast recording device, such as Figure 2 As shown, including:

[0138] The live broadcast recording module 1 is used to record the live broadcast video until the live broadcast ends when the first user starts the live broadcast teaching in the classroom;

[0139] Video preprocessing module 2, used to preprocess the live video to obtain recorded teaching lessons;

[0140] The recording and broadcasting publishing module 3 is used to publish the recording and broadcasting teaching courses.

[0141] The video preprocessing module 2 preprocesses the live video to obtain a recorded teaching course, including:

[0142] Identify segments to be removed from live video;

[0143] Remove the segments that need to be removed from the live video;

[0144] The live video is packaged into a course after the segments to be removed to obtain the recorded teaching course.

[0145] The video preprocessing module 2 identifies the segments to be removed in the live video, including:

[0146] Identify unvoiced segments from live video;

[0147] If the non-voice segment meets the first active silence condition, analyzing the start and end time periods of the non-voice segment;

[0148] Obtaining a movement trajectory map of the first user in the classroom during the start and end time periods;

[0149] If the moving trajectory diagram meets the second active silent condition, the segment without voice is regarded as a segment to be removed;

[0150] Among them, the first active silent condition includes:

[0151] The duration of the segment without vocalization exceeds the first duration threshold;

[0152] Moreover, the vocal semantic sets in the adjacent vocal segments before and after the non-vocal segments in the live video do not match multiple standard vocal semantic sets;

[0153] Moreover, the action sets of people in the segments without voice and the adjacent segments with voice before and after the segments without voice in the live video do not match the multiple standard action sets of people;

[0154] Among them, the second active silent condition includes:

[0155] The ratio of the target local trajectory segment to the total movement trajectory on the movement trajectory diagram exceeds the ratio threshold; wherein the relative position relationship between each trajectory point on the target local trajectory segment and the live broadcast camera position matches the standard relative position relationship.

[0156] After the recording and broadcasting publishing module 3 records and broadcasts the recorded and broadcasted teaching course, it also includes:

[0157] The interactive playback module is used to interactively play the recorded teaching course to the second user when the second user requests the recorded teaching course.

[0158] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A classroom teaching live broadcast and recording method, characterized in that: include: When the first user starts a live teaching session in the classroom, the live video is recorded until the end of the live session; Pre-process the live video to obtain recorded teaching lessons; Record and publish recorded teaching courses; The preprocessing of the live video to obtain the recorded teaching course includes: Identify segments to be removed from live video; Remove the segments that need to be removed from the live video; After removing the segments to be removed, the live video is packaged into a course to obtain a recorded teaching course; The identifying the segments to be removed from the live video includes: Identify unvoiced segments from live video; If the unvoiced segment meets the first active silence condition, the start and end time periods of the unvoiced segment are analyzed; if the unvoiced segment meets the first active silence condition, it is preliminarily determined that the unvoiced segment is generated by the first user in the case of active silence; active silence means that the first user actively chooses silence and subjectively does not want the corresponding part of the unvoiced segment to be included in the recorded teaching course; Obtaining a movement trajectory map of the first user in the classroom during the start and end time periods; If the moving trajectory graph meets the second active silence condition, the segment without voice is regarded as a segment to be removed; when the moving trajectory graph meets the second active silence condition, it means that it is completely determined that the segment without voice is generated by the first user in the active silence condition; Among them, the first active silent condition includes: The duration of the segment without vocalization exceeds the first duration threshold; Moreover, the vocal semantic sets in the adjacent vocal segments before and after the non-vocal segments in the live video do not match multiple standard vocal semantic sets; Moreover, the action sets of people in the segments without voice and the adjacent segments with voice before and after the segments without voice in the live video do not match the multiple standard action sets of people; Among them, the second active silent condition includes: The ratio of the target local trajectory segment to the total movement trajectory on the movement trajectory diagram exceeds the ratio threshold; wherein the relative position relationship between each trajectory point on the target local trajectory segment and the live broadcast camera position matches the standard relative position relationship.

2. The classroom teaching live broadcast and recording method according to claim 1, characterized in that: After the recorded teaching course is recorded and released, the method further includes: When the second user orders the recorded teaching course, the recorded teaching course is interactively played to the second user.

3. The classroom teaching live broadcast and recording method according to claim 2, characterized in that: The interactively playing the recorded teaching course to the second user includes: Displaying the recorded teaching course to the second user, and controlling the displayed recorded teaching course to start playing; When the second user views the displayed recorded teaching course and the target content at the same time, if the target content meets the content linkage triggering condition, based on the first remaining playback time axis of the recorded teaching course and the second remaining playback time axis of the target content, the content linkage resources and the corresponding resource arrangement time period on the first remaining playback time axis are determined; Arrange content linkage resources within the resource arrangement time period; Based on the first remaining playback time axis after the content linkage resource arrangement is completed within the resource arrangement time period, controlling the displayed recorded teaching course to continue to be played; The content linkage trigger conditions include: The cumulative time duration during which the target content is viewed by the second user and is not moved by the second user exceeds a second time duration threshold; Also, the content type of the target content matches the standard content type.

4. The classroom teaching live broadcast and recording method as claimed in claim 3, characterized in that: The determining of the content linkage resources and the corresponding resource arrangement time period on the first remaining playback time axis based on the first remaining playback time axis of the recorded teaching course and the second remaining playback time axis of the target content includes: Based on the content pairing constraint, the first remaining playback time axis and the second remaining playback time axis are content-paired to obtain a paired first content item and a second content item; wherein the first content item is from a first playback time period on the first remaining playback time axis, and the second content item is from a second playback time period on the second remaining playback time axis; Extracting features from the first content item and the second content item to obtain a content feature set; Generate resource screening conditions based on content feature sets; Based on the resource screening conditions, content linkage resources are screened out from the content linkage resource library; Using the overlapping time period between the first play time period and the second play time period as the resource arrangement time period; Among them, content matching constraints include: The association relationship between the first content item and the second content item matches the standard association relationship; Furthermore, a start time difference and an end time difference between the first playback time period and the second playback time period do not exceed a time difference threshold.

5. A classroom teaching live broadcast and recording device, characterized in that: include: A live broadcast recording module, used for recording the live broadcast video until the end of the live broadcast when the first user starts the live teaching in the classroom; The video preprocessing module is used to preprocess the live video to obtain the recorded teaching course; The recording and broadcasting publishing module is used to publish the recording and broadcasting teaching courses; The video preprocessing module preprocesses the live video to obtain a recorded teaching course, including: Identify segments to be removed from live video; Remove the segments that need to be removed from the live video; After removing the segments to be removed, the live video is packaged into a course to obtain a recorded teaching course; The video preprocessing module identifies the segments to be removed in the live video, including: Identify unvoiced segments from live video; If the unvoiced segment meets the first active silence condition, the start and end time periods of the unvoiced segment are analyzed; if the unvoiced segment meets the first active silence condition, it is preliminarily determined that the unvoiced segment is generated by the first user in the case of active silence; active silence means that the first user actively chooses silence and subjectively does not want the corresponding part of the unvoiced segment to be included in the recorded teaching course; Obtaining a movement trajectory map of the first user in the classroom during the start and end time periods; If the moving trajectory graph meets the second active silence condition, the segment without voice is regarded as a segment to be removed; when the moving trajectory graph meets the second active silence condition, it means that it is completely determined that the segment without voice is generated by the first user in the active silence condition; Among them, the first active silent condition includes: The duration of the segment without vocalization exceeds the first duration threshold; Moreover, the vocal semantic sets in the adjacent vocal segments before and after the non-vocal segments in the live video do not match multiple standard vocal semantic sets; Moreover, the action sets of people in the segments without voice and the adjacent segments with voice before and after the segments without voice in the live video do not match the multiple standard action sets of people; Among them, the second active silent condition includes: The ratio of the target local trajectory segment to the total movement trajectory on the movement trajectory diagram exceeds the ratio threshold; wherein the relative position relationship between each trajectory point on the target local trajectory segment and the live broadcast camera position matches the standard relative position relationship.

6. The classroom teaching live broadcast and recording device according to claim 5, characterized in that: After the recording and broadcasting publishing module records and broadcasts the recording and broadcasting teaching course, it also includes: The interactive playback module is used to interactively play the recorded teaching course to the second user when the second user requests the recorded teaching course.

Images