A method for automatically cutting and generating dubbing materials from English movies based on knowledge points

Abstract

This invention discloses a method for automatically cutting and generating dubbing materials from English movies based on knowledge points, which includes the following steps: (1) the movie is entered into a database and movie subtitles are collected via the network; (2) the movie is translated by AI to obtain AI subtitles, and the similarity is compared with multiple collected movie subtitles to obtain more accurate subtitles, and the subtitle timeline is corrected and confirmed; (3) the subtitle file is divided into multiple segments according to the English knowledge points of primary and secondary schools; (4) the movie is cut according to the segmented subtitle segments to obtain movie video segments corresponding to each subtitle segment; (5) the audio is separated from each video segment, and the audio file is separated into human voice and background sound to obtain the background sound and original sound required for dubbing materials; (6) the background sound and subtitles of the video segments are synthesized and transcoded to obtain the final dubbing materials.

Description

Technical Field

[0001] This invention relates to dubbing technology, and more particularly to automatic dubbing material technology. Background Technology

[0002] As an innovative foreign language learning method, software and websites like FunDubbing provide learners with special video clips. These clips are typically taken from classic foreign language films or TV series, and the audio and background have been pre-separated. Chinese and foreign language subtitles are also provided. Learners can dub these clips themselves and learn from the content through this method. Compared to the passive, listening-based traditional foreign language learning method, this active and immersive approach effectively increases learner engagement and allows them to more effectively master the foreign language material contained in the dubbed content.

[0003] However, preparing such teaching dubbing materials requires professionals proficient in foreign languages ​​and video processing technologies, consuming a significant amount of time for video editing, original audio separation, and foreign language subtitle generation, resulting in high costs, long processing times, and difficulty in large-scale production. Furthermore, for primary and secondary school students, how to automatically generate dubbing materials from English movies that match the knowledge points in primary and secondary school English teaching is also a technology that urgently needs to be developed by technicians in this field. Summary of the Invention

[0004] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is to automatically cut out the required video segments and subtitles from a complete movie based on the knowledge points of primary and secondary schools, separate the human voice and background from the video segments, and transcode the obtained video into dubbing material.

[0005] To achieve the above objectives, this invention first provides a method for automatically cutting and generating dubbing materials from English movies based on knowledge points, which includes the following steps:

[0006] (1) Movies are entered into the database, and movie subtitles are collected via the internet;

[0007] (2) Perform AI translation on the movie to obtain AI subtitles, compare the similarity with multiple movie subtitles collected, obtain more accurate subtitles, and correct and confirm the subtitle timeline;

[0008] (3) Based on the English knowledge points in primary and secondary schools, the subtitle file is divided into multiple segments;

[0009] (4) Based on the segmented subtitle segments, the movie is cut to obtain movie video clips corresponding to each subtitle segment;

[0010] (5) Extract audio from each video segment and separate human voice and background sound from the audio file to obtain the background sound and original sound required for dubbing material;

[0011] (6) Combine background audio and subtitles in the video clips and transcode them to obtain the final dubbing material.

[0012] Furthermore, it also includes step (7), which involves uploading the dubbing material to the dubbing platform APP.

[0013] Furthermore, in step (4), the ffmpeg method is used for online visual video editing.

[0014] Furthermore, in step (5), the Spleeter method is used to separate human voices from background noise.

[0015] Furthermore, the transcoded video materials include standard definition, high definition, Blu-ray, and ultra-high definition at multiple bitrates.

[0016] This invention also provides a system for automatically segmenting and generating dubbing materials from English movies based on knowledge points, including a subtitle acquisition module, a subtitle comparison and selection module, a subtitle timeline correction module, a subtitle segmentation module, a video segmentation module, an audio separation module, and a video synthesis and transcoding module. The subtitle acquisition module is used to collect movie subtitles from a database via the network; the subtitle comparison and selection module is used to perform AI audio translation on the movie, obtain AI subtitles, and compare their similarity with multiple collected movie subtitles to obtain more accurate subtitles; the subtitle timeline correction module is used to correct and confirm the subtitle timeline; the subtitle segmentation module is used to segment the subtitle file into multiple segments based on primary and secondary school English knowledge points; the video segmentation module is used to segment the movie based on the segmented subtitle segments to obtain movie video clips corresponding to each subtitle segment; the audio separation module is used to separate audio from each video clip and separate human voice and background sound from the audio file to obtain the background sound and original sound required for the dubbing material; the video synthesis and transcoding module is used to synthesize and transcode the background sound and subtitles from the video clips to obtain the final dubbing material.

[0017] Furthermore, the video segmentation module uses the ffmpeg method for online visual video editing.

[0018] Furthermore, the audio separation module uses the spleeter method to separate human voices from background noise.

[0019] Furthermore, the system is configured to upload the generated voice-over materials to the voice-over platform app.

[0020] Furthermore, the transcoded video materials include standard definition, high definition, Blu-ray, and ultra-high definition at multiple bitrates.

[0021] This invention can automatically acquire movie subtitles from the internet, eliminating the time and cost of manually searching for subtitle files; it enables large-scale production of dubbing materials, greatly enriching the dubbing resources for "English Fun Dubbing"; users can easily proofread and improve subtitles online; and through automated movie cutting, it significantly saves company resources and production costs.

[0022] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0023] Figure 1 This is a flowchart of a preferred embodiment of the present invention;

[0024] Figure 2 This is a diagram of the system operation interface for subtitle processing in a preferred embodiment of the present invention;

[0025] Figure 3 This is a diagram of the system operation interface for video editing in a preferred embodiment of the present invention;

[0026] Figure 4 This is a diagram of the system operation interface for audio separation in a preferred embodiment of the present invention;

[0027] Figure 5 This is a diagram of the operation interface for supplementing information in a video in a preferred embodiment of the present invention. Detailed Implementation

[0028] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0029] like Figure 1 As shown, the method for automatically cutting and generating dubbing materials from English videos based on knowledge points according to the present invention includes the following steps:

[0030] (1) Add the required movies to the database and collect the subtitle files that match the movies. Subtitle files can be automatically collected via the network or uploaded manually. The user interface is as follows: Figure 2 As shown.

[0031] (2) The film is transcribed using AI to obtain AI subtitles, which are then compared with multiple collected subtitle files to obtain more accurate subtitles. The subtitle timeline is then corrected. In a preferred embodiment, this invention employs intelligent speech recognition technology based on Alibaba. The system operation interface is as follows: Figure 3 As shown.

[0032] (3) After the subtitles are confirmed, the subtitle file is pre-segmented into multiple subtitle segments based on the knowledge points for primary and secondary schools. The knowledge points for primary and secondary schools can be determined according to the teaching syllabus of the textbooks, or by the instructors themselves. In a preferred embodiment, this invention employs online visual video editing technology based on ffmpeg.

[0033] (4) Edit the video based on the subtitle segments, separate the audio from the video segments, and determine the final video segments. In a preferred embodiment, the present invention employs Spleeter-based human voice and background noise separation technology. The system operation interface is as follows: Figure 4 As shown.

[0034] (5) Add information to video clips. The system interface is as follows: Figure 5 As shown.

[0035] (6) Transcode and watermark each video clip to complete the dubbing material.

[0036] (7) Upload it to the resource library for app users to use for dubbing.

[0037] This invention can automatically acquire movie subtitles from the internet, eliminating the time and cost of manually searching for subtitle files; it enables large-scale production of dubbing materials, greatly enriching the dubbing resources for "English Fun Dubbing"; users can easily proofread and improve subtitles online; and through automated movie cutting, it significantly saves company resources and production costs.

[0038] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for automatically segmenting English movies into dubbing materials based on knowledge points, characterized in that, Including the following steps: (1) Movies are entered into the database, and movie subtitles are collected via the internet; (2) Perform AI translation on the movie to obtain AI subtitles, compare the similarity with multiple movie subtitles collected, obtain more accurate subtitles, and correct and confirm the subtitle timeline; (3) Based on the English knowledge points in primary and secondary schools, the subtitle file is divided into multiple segments; (4) Based on the segmented subtitle segments, the movie is cut to obtain movie video clips corresponding to each subtitle segment; (5) Extract audio from each video segment and separate human voice and background sound from the audio file to obtain the background sound and original sound required for dubbing material; (6) Combine background audio and subtitles in the video clips and transcode them to obtain the final dubbing material; The English knowledge points for primary and secondary schools can be determined according to the teaching syllabus of the textbooks, or they can be determined by the teachers themselves.

2. The method for automatically cutting and generating dubbing materials from English movies based on knowledge points as described in claim 1, wherein, It also includes step (7), which involves uploading the dubbing material to the dubbing platform APP.

3. The method for automatically cutting and generating dubbing materials from English movies based on knowledge points as described in claim 1, wherein, The transcoded video footage includes standard definition, high definition, Blu-ray, and ultra-high definition at multiple bitrates.

4. The method for automatically cutting and generating dubbing materials from English movies based on knowledge points as described in claim 1, wherein, In step (4), the ffmpeg method is used for online visual video editing.

5. The method for automatically cutting and generating dubbing materials from English movies based on knowledge points as described in claim 1, wherein, In step (5), the Spleeter method is used to separate human voice from background noise.

6. A system for automatically cutting and generating dubbing materials from English movies based on knowledge points, characterized in that, The system includes a subtitle acquisition module, a subtitle comparison and selection module, a subtitle timeline correction module, a subtitle segmentation module, a video segmentation module, an audio separation module, and a video synthesis and transcoding module. The subtitle acquisition module collects movie subtitles from a database via the network. The subtitle comparison and selection module performs AI-based audio translation of the movie to obtain AI subtitles, which are then compared with multiple collected movie subtitles to obtain more accurate subtitles. The subtitle timeline correction module corrects and confirms the subtitle timeline. The subtitle segmentation module divides the subtitle file into multiple segments based on primary and secondary school English knowledge points. The video segmentation module cuts the movie based on the segmented subtitle segments to obtain movie video clips corresponding to each subtitle segment. The audio separation module separates the audio from each video clip and separates the human voice and background sound from the audio file to obtain the background sound and original sound required for dubbing. The video synthesis and transcoding module synthesizes and transcodes the background sound and subtitles from the video clips to obtain the final dubbing material.

7. The system for automatically cutting and generating dubbing materials from English movies based on knowledge points as described in claim 6, wherein, The video segmentation module uses ffmpeg for online visual video editing.

8. The system for automatically cutting and generating dubbing materials from English movies based on knowledge points as described in claim 6, wherein, The audio separation module uses the spleeter method to separate human voices from background noise.

9. The system for automatically cutting and generating dubbing materials from English movies based on knowledge points as described in claim 6, wherein, The system is configured to upload the generated voice-over materials to the voice-over platform app.

10. The system for automatically cutting and generating dubbing materials from English movies based on knowledge points as described in claim 6, wherein, The transcoded video footage includes standard definition, high definition, Blu-ray, and ultra-high definition at multiple bitrates.

Images