A method for editing teaching videos based on dynamic text generation

Abstract

A method for dynamically generating teaching videos based on text, the main steps of which include: obtaining text input by a user, performing NLP processing on the text, retrieving teaching videos, converting meta video data into audio and video, calculating similarity features of meta video data, user selecting audio and video materials, and synthesizing virtual human teaching examples. The present invention can dynamically generate teaching videos based on text, automatically create virtual demonstrations, so as to lower the technical threshold of video production, reduce the time and manpower cost of video production; and can quickly find suitable demonstration materials, improve the efficiency of example retrieval, and enhance the teaching experience of teaching users.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an editing method for dynamically generating teaching videos based on text. Background Art

[0002] Today, the technology of virtual classroom learning is very mature. Teachers can enter the established virtual classrooms through the Internet to teach, and students can enter the virtual classrooms that match their learning progress through the Internet to study. A virtual classroom can accommodate multiple students at the same time.

[0003] The invention patent with publication number CN202111338371.7 discloses an automated teaching video generation method, device, computer equipment and storage medium; the method includes: obtaining the original video from the terminal; processing the original video to form a number of video material files and corresponding video tags; using the video tags to perform custom assembly of a number of video material files to form character video materials; obtaining candidate scenes; obtaining voices from the terminal and storing the voices in a database: retrieving and determining the voices to be synthesized from the database: synthesizing according to the character video materials, candidate scenes and the voices to be synthesized to obtain a teaching video. This patent can meet the production of a large number of teaching videos in a short period of time, so as to save the work of schools and teachers in recording and producing teaching videos, reduce the workload of teachers, and reduce the human resource costs of schools, and can customize images and voices.

[0004] However, the above patent requires users to provide original videos and voices, and cannot automatically generate teaching content. In addition, the teaching data annotation method in the patent is time-consuming and labor-intensive, requiring a lot of manual participation and review, and it is difficult to ensure that the annotation is accurate.

[0005] Quality and efficiency. Summary of the invention

[0006] In response to the above-mentioned technical problems, this technical solution provides an editing method for dynamically generating teaching videos based on text, using generative adversarial network technology to automatically generate teaching content without the need for users to provide original videos and voices; by using natural language processing technology, teaching data is automatically annotated to avoid a large amount of manual participation and review, thereby improving the quality and efficiency of annotation; it can effectively solve the above-mentioned problems.

[0007] The present invention is achieved through the following technical solutions:

[0008] A method for dynamically generating teaching videos based on text, the main steps of which include obtaining text input by a user, performing NLP processing on the text, retrieving teaching videos, converting original video data into audio and video, calculating similarity features of the original video data, the user selecting audio and video materials, and synthesizing virtual human teaching examples; the method specifically includes the following steps:

[0009] Step 1: Obtain video material dataset D;

[0010] Step 2: Build an initial audio and video synthesis model;

[0011] Step 3: Auxiliary training data set model T and preprocessing;

[0012] Step 4: Retrieve audio and video features and scenes; the specific method is:

[0013] Step 4.1: First, obtain the text input by the user, and use Jieba word segmentation to segment the text keywords to obtain the user search data set D3; then calculate the original video data similarity feature of the data set model T generated by the original video data used in step 3 to generate a visual list of 1-6 materials;

[0014] Step 4.2: Retrieve the data set model T in step 3, calculate the similarity features of the original video data, and generate a list of 1-6 visual materials: audio and video course clip data L={l1,l2,…l6};

[0015] Step 5: Based on the list of audio and video course segment data L formed in step 4.2, the user makes a single or multiple selection;

[0016] Step 6: Synthesize a virtual human teaching example video;

[0017] Step 7: Save the teaching example video in step 6 back to the video material dataset D.

[0018] Furthermore, the video material described in step 1 includes two parts: the first part is the recorded teaching audio and video clips; the second part is the video clips obtained by replaying the generated virtual human teaching audio and video clips.

[0019] Furthermore, the construction of the initial audio and video synthesis model described in step 2 is specifically performed as follows:

[0020] Step 2.1: Process the video continuous speech recognition through the hidden Markov chain (HMM) method to obtain the text dataset D1;

[0021] Step 2.2: Use Jieba word segmentation and Baidu NLP API to process text NLP and perform word segmentation to obtain the text word segmentation dataset D2;

[0022] Step 2.3: Store the video text dataset D1-D2.

[0023] Furthermore, the auxiliary training data set model T described in step 3 is preprocessed, and the specific method is as follows:

[0024] Step 3.1: Traverse the video text dataset D1-D2 stored in step 2.3 and extract the text word sequence and text label model in the video;

[0025] Step 3.2: Construct a metadata description corresponding to a video. The metadata can describe the encoding parameters; accessing this information through metadata is faster than through media type attributes.

[0026] Step 3.3: Generate a metadata description set model, which contains descriptive information of the media content, such as title, keyword, category, time sequence and scene.

[0027] Furthermore, the synthesized virtual human teaching example video described in step 6 is generated by integrating the audio and video materials selected by the user with the virtual human to generate a teaching example video E with a virtual human anchor and a text theme. The specific operation steps are:

[0028] Step 6.1: Get a pre-set virtual human image and use it as input;

[0029] Step 6.2: Use the Wave2Lip algorithm to build the LipGAN model, convert text into speech, and generate a virtual human lip movement video sequence. Beneficial Effects

[0030] The editing method for dynamically generating teaching videos based on text proposed in the present invention has the following beneficial effects compared with the prior art:

[0031] (1) The present invention can dynamically generate teaching videos based on text and automatically create virtual demonstrations to lower the technical threshold for video production and reduce the time and manpower costs of video production. By using generative adversarial network (GAN) technology, teaching content is automatically generated without the need for users to provide original videos and voices; by using natural language processing (NLP) technology, teaching data is automatically annotated to avoid a large amount of manual participation and review, thereby improving the quality and efficiency of annotation. At the same time, this method only requires general-purpose teaching equipment and a single video acquisition system to provide implementation support, reducing implementation costs and complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0033] Figure 2 This is the overall flow chart of the text segmentation method based on hidden Markov chain in step 2.

[0034] Figure 3 This is a flow chart of the text similarity feature calculation method in step 4.

[0035] Figure 4 This is the material list diagram visualized in step 4.

[0036] Figure 5 This is the flowchart of the Wave2Lip algorithm in step 6. DETAILED DESCRIPTION

[0037] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Under the premise of not departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solutions of the present invention should all fall within the protection scope of the present invention. Example

[0038] like Figure 1 As shown, an editing method for dynamically generating teaching videos based on text, the main steps include obtaining text input by the user, performing NLP processing on the text, retrieving teaching videos, converting the original video data into audio and video, calculating the similarity features of the original video data, the user selecting audio and video materials, and synthesizing virtual human teaching examples; specifically, the following steps are included:

[0039] Step 1: Obtain a video material dataset D; the video material includes two parts: the first part is the recorded teaching audio and video clips; the second part is the video clips obtained by retrieving the generated virtual human teaching audio and video clips.

[0040] Step 2: Build the initial audio and video synthesis model; the specific operation method is:

[0041] Step 2.1: Process the video continuous speech recognition by the hidden Markov chain (HMM) method. The specific processing steps are as follows: Figure 2 As shown, the text dataset D1 is obtained.

[0042] Step 2.2: Perform word segmentation on the text by mixing Jieba word segmentation and Baidu NLP API to obtain the text word segmentation dataset D2.

[0043] Step 2.3: Store the video text dataset D1-D2.

[0044] Step 3: Auxiliary training data set model T and preprocessing; the specific method is:

[0045] Step 3.1: Traverse the video text dataset D1-D2 stored in step 2.3 and extract the text word sequence and text label model in the video.

[0046] Step 3.2: Construct a metadata description corresponding to a video. The metadata can describe the encoding parameters; accessing this information through metadata is faster than through media type attributes.

[0047] Step 3.3: Generate a dataset model T, which contains descriptive information of media content such as title, keyword, category, time series, and scene.

[0048] Step 4: Retrieve audio and video features and scenes; the specific method is:

[0049] Step 4.1: First, obtain the text input by the user, use Jieba word segmentation to segment the text keywords, and obtain the user search data set D3; then calculate the original video data similarity features of the data set model T generated by the original video data used in step 3, and generate a visual list of 1-6 materials.

[0050] Step 4.2: Retrieve the dataset model T in step 3 and calculate the similarity features of the original video data. The specific method of calculating the similarity features is as follows: Figure 3 As shown; and generate a visualization 1-6 material list: audio and video course segment data L={l1,l2,…l6}; the visualization material list is as follows Figure 4 shown.

[0051] Step 5: Based on the list of audio and video course segment data L formed in step 4.2, the user makes single or multiple selections.

[0052] Step 6: Synthesize a virtual human teaching example video; this is to integrate the audio and video materials selected by the user with the virtual human to generate a teaching example video E with a virtual human anchor and a text theme. The specific operation steps are as follows:

[0053] Step 6.1: Get a pre-set virtual human image and use it as input.

[0054] Step 6.2: Use the Wave2Lip algorithm. The operation of the Wave2Lip algorithm is as follows: Figure 5 As shown in the figure, a LipGAN model is established to convert text into speech and generate a video sequence of virtual human lip movement.

[0055] Step 7: Save the teaching example video in step 6 back to the video material dataset D.

[0056] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes, substitutions and improvements within the technical scope disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A method for dynamically generating teaching videos based on text, characterized in that: The main steps include obtaining text input by the user, performing NLP processing on the text, retrieving teaching videos, converting the original video data into audio and video, calculating the similarity features of the original video data, the user selecting audio and video materials, and synthesizing virtual human teaching examples; the specific steps include the following: Step 1: Obtain video material dataset D; Step 2: Build an initial audio and video synthesis model; Step 3: Auxiliary training data set model T and preprocessing; Step 4: Retrieve audio and video features and scenes; the specific method is: Step 4.1: First, obtain the text input by the user, and use Jieba word segmentation to segment the text keywords to obtain the user search data set D3; then calculate the original video data similarity feature of the data set model T generated by the original video data used in step 3 to generate a visual list of 1-6 materials; Step 4.2: Retrieve the data set model T in step 3, calculate the similarity features of the original video data, and generate a list of 1-6 visual materials: audio and video course clip data L={l1,l2,…l6}; Step 5: Based on the list of audio and video course segment data L formed in step 4.2, the user makes a single or multiple selection; Step 6: Synthesize a virtual human teaching example video; Step 7: Save the teaching example video in step 6 back to the video material dataset D.

2. The editing method for dynamically generating teaching videos based on text according to claim 1, characterized in that: The video material described in step 1 includes two parts: the first part is the recorded teaching audio and video clips; the second part is the video clips obtained by replaying the generated virtual human teaching audio and video clips.

3. The editing method for dynamically generating teaching videos based on text according to claim 1, characterized in that: The specific operation method of constructing the initial audio and video synthesis model described in step 2 is as follows: Step 2.1: Process the video continuous speech recognition through the hidden Markov chain (HMM) method to obtain the text dataset D1; Step 2.2: Use Jieba word segmentation and Baidu NLP API to process text NLP and perform word segmentation to obtain the text word segmentation dataset D2; Step 2.3: Store the video text dataset D1-D2.

4. The editing method for dynamically generating teaching videos based on text according to claim 3 is characterized in that: The auxiliary training data set model T described in step 3 is preprocessed, and the specific method is as follows: Step 3.1: Traverse the video text dataset D1-D2 stored in step 2.3 and extract the text word sequence and text label model in the video; Step 3.2: Construct a metadata description corresponding to a video. The metadata can describe the encoding parameters; accessing this information through metadata is faster than through media type attributes; Step 3.3: Generate a metadata description set model, which contains descriptive information of the media content, including title, keyword, category, time series and scene.

5. The editing method for dynamically generating teaching videos based on text according to claim 1 is characterized in that: The synthetic virtual human teaching example video described in step 6 is generated by integrating the audio and video materials selected by the user with the virtual human to generate a teaching example video E with a virtual human anchor and a text theme. The specific operation steps are as follows: Step 6.1: Get a pre-set virtual human image and use it as input; Step 6.2: Use the Wave2Lip algorithm to build the LipGAN model, convert text into speech, and generate a virtual human lip movement video sequence.

Images