A method and system for automatic generation of a lecture video based on document content

Abstract

The application discloses a kind of based on document content's explanation video automation generation method and system, it is related to online education and multimedia automation generation technical field.The method will source document to the production flow of teaching video be divided into three core stages of outline generation, HTML and beat sequence joint generation, splicing and check, and four automatic construction steps of TTS dubbing and time axis construction, browser automation screen recording, audio and video synthesis, form complete end-to-end pipeline.The application solves the problems of low production efficiency, sound and picture out of synchronization, fixed style and unreasonable content layout of traditional teaching video production by HTML and beat sequence joint generation mechanism, three-layer sound and picture synchronization architecture, multi-style animation engine, multi-style theme system and multi-level canvas filling adaptive rules, shortens the production cycle from several days to several weeks to minute level, while ensuring the animation expressiveness, style customizability and cross-platform rendering consistency of the video.

Description

Technical Field

[0001] This invention belongs to the field of online education and multimedia automated generation technology, specifically relating to a method and system for automatically generating explanatory videos based on document content. Background Technology

[0002] With the rapid development of online education and the paid knowledge industry, the demand for converting text lecture notes into video courses has increased dramatically. However, existing technologies have the following shortcomings: (1) Fragmented production process and high labor costs: The traditional process requires script writing, PPT design and production, audio recording and dubbing, video editing and synthesis, etc., each step depends on different professional tools and personnel. A 30-minute course video usually requires several days to several weeks of work.

[0003] (2) It is difficult to accurately synchronize animation and narration: In existing tools, the visual animation and audio narration are usually produced separately. The timing needs to be manually aligned during the synthesis stage. There is a lack of accurate frame-level synchronization mechanism, which easily leads to audio-visual desynchronization.

[0004] (3) Fixed style and lack of personalized customization: The existing automatic generation scheme uses a fixed template, and users cannot customize the visual style (color scheme, font, animation effect, etc.), which makes it difficult to meet the branding or personalization needs.

[0005] (4) Lack of flexible review and modification mechanism: The fully automated generation cannot be previewed or intervened by users before completion. If it does not meet expectations, it can only be done again, resulting in high trial and error costs.

[0006] (5) Insufficient or overflowing content: When converting document content into a fixed-size (1920×1080) slide, the lack of intelligent assessment and adaptive mechanism for content density can easily lead to large areas of blank space or content overflow during cropping.

[0007] Existing automatic video generation methods mainly fall into two categories: one is simple generation based on TTS combined with static images, lacking animation expressiveness; the other is content filling based on preset video templates, lacking the ability to adapt to diverse document structures. Neither of these methods can simultaneously meet the comprehensive requirements of rich animation, synchronized audio and video, customizable style, adaptive content, and support for human intervention.

[0008] Traditional course video production requires multiple steps, including scriptwriting, PPT design, audio recording and narration, and video editing and compositing. Each step relies on different professional tools and personnel, resulting in a production cycle of several days to weeks. Existing automated solutions either only generate static text and image videos, lacking animation capabilities, or rely on fixed templates, failing to adapt to diverse document structures and personalized style requirements. Furthermore, precise synchronization between visual animation and audio narration, adaptive layout density of generated content, and support for flexible user review and modification are all pain points of existing technologies. Summary of the Invention

[0009] To address the technical challenges of automating the end-to-end generation of instructional videos from documents, including low synchronization accuracy between animation and narration, insufficient animation expressiveness, weak style customization capabilities, unreasonable content layout, and lack of intermediate review mechanisms in existing technologies, this application designs a method and system for automatically generating instructional videos based on document content. Through innovative methods such as an end-to-end pipeline architecture, a synchronization mechanism for jointly generating HTML and beat sequences, a multi-style theme system, and a user-defined template mechanism, the system allows users to provide only a text document. The system can automatically understand the document structure, break it down into logical slides, automatically design aesthetically pleasing page layouts and animation effects, automatically generate AI voice-over, and ensure precise synchronization between narration and screen transitions. Finally, it outputs a complete 1080p instructional video with a single click.

[0010] A method for automatically generating explanatory videos based on document content includes the following steps: Step S1, Outline Generation Stage: Perform structured parsing on the input source document, identify the document's heading levels, paragraphs, and list structure, automatically divide the content into 8-18 teaching slide units, label each slide unit with content type, rhythm, and visual layout suggestions, and generate a structured outline file; Step S2, Joint Generation Stage: For each slide unit, simultaneously generate HTML content containing layout, style, animable elements and their globally unique identifier IDs, as well as a beat sequence containing explanatory text, animation actions, target element selectors and action parameters, to achieve joint output of HTML content and beat sequence; Preferably, the joint generation adopts a single-call multi-task output mechanism of a large language model: by constructing a compound prompt containing HTML element ID naming conventions and CSS selector mapping requirements, the model is required to output HTML code and JSON format beat sequences simultaneously in a single response.

[0011] Specifically, the core requirements for the prompt words include: (1) HTML element IDs follow the unified naming convention of “s{slide number}-{element type}-{serial number / name}”, where element types include title (title), subtitle (subtitle), bullet (list item), text (paragraph), card (card), node (process node), stat (number), kw (keyword), etc., such as s5-title, s3-card-1, s7-bullet-3; (2) The target field of each animation action in the beat sequence uses a CSS selector (such as #s5-card-1) to precisely point to the element with the corresponding ID in the HTML; (3) The length of the explanatory text (text field) for each beat is 30-50 Chinese characters; (4) Each slide contains 4-12 beats. The first beat is a transition action (page-in or section-change). The same action is not repeated more than 3 times. At least 3 different action types are used. (5) The output is in JSON format, containing two fields: "html" and "beats". "html" is the complete section HTML string, and "beats" is an array of beat objects. Each beat object contains the fields: text (explanation text), action (action name), target (CSS selector), and params (action parameters, optional).

[0012] The example structure of the prompt is: "Please generate HTML slides and corresponding beat sequences for the following teaching content: [Insert key points and explanation summary from the outline here]. Strictly adhere to the following requirements for output: HTML element IDs follow the naming convention s{slide number}-{element type}-{serial number}; beat targets use the corresponding CSS selectors; each beat's explanation text is 30-50 words; each page has 4-12 beats; the output JSON object contains two fields: html and beats." This mechanism ensures the natural consistency between HTML elements and animation target selectors in the beat sequence from the generation source, eliminating the need for subsequent manual alignment steps. In contrast, existing technologies treat HTML page generation and animation timing arrangement as two independent steps, requiring manual mapping of animation targets to HTML elements, which is inefficient and prone to errors.

[0013] Step S3, splicing and verification stage: splice the HTML content and beat sequence of all slides into a complete page, and perform automated verification of ID consistency check, text length verification, continuous action detection, and layout density self-check; Step S4, TTS Dubbing and Timeline Construction: Using a cloud-based neural network speech synthesis engine, the narration text for each beat is segmented and synthesized into audio. The global start time is calculated based on the actual duration of each audio segment, and the timeline is constructed and injected into the HTML page. Step S5: Automated screen recording in a browser: Load an HTML page containing a timeline using a headless browser, and drive page switching and animation execution precisely according to the timeline using a controller, while simultaneously recording a 1920×1080 resolution screen output video stream. Step S6, Audio and Video Synthesis: Extract segments that match the audio duration from the screen recording video stream, transcode them to H.264 encoding format, merge the AAC audio, and output the final teaching video file.

[0014] Preferably, in step S2, all animable elements in the HTML follow the globally unique ID naming convention of s{slide number}-{element type}-{serial number / name}; each animation action in the beat sequence is precisely pointed to the corresponding ID in the HTML through a CSS selector, ensuring the consistency between the animation target and the HTML element.

[0015] Preferably, in step S5, the animation execution is based on a multi-style animation engine. The multi-style animation engine defines 15 standardized actions in four major categories: transition, entrance, emphasis, and variation. Each entrance action supports 6 selectable animation styles, forming a 15×6 animation combination matrix. It is driven by GSAP and incorporates up to 8% random jitter to simulate the rhythm of natural hand movements.

[0016] Preferably, the random jitter is achieved by applying proportional noise to the duration parameter of the animation: specifically, for each animation action, its actual execution duration d' = d × (1 + (rand - 0.5) × 2 × 0.08), where d is the preset duration and rand is a uniformly distributed random number in the interval [0, 1), causing the actual duration to fluctuate randomly within ±8% of the preset value. This mechanism is applied to the duration parameter of all entrance animations (including five styles: float-up, wipe-right, scale-bounce, fade-only, and pop-in) and transition animations (page-in, section-change), avoiding the mechanical repetition of animation rhythms across slides and making the viewing experience closer to manually produced videos.

[0017] Preferably, before step S1, a user preference configuration step is also included, which allows users to select a preset visual theme, adaptively match the style through document theme words, or customize the style by describing it in natural language and specifying a reference HTML template; the system keeps the layout class name structure unchanged and only modifies the CSS design token to reproduce the target style.

[0018] Preferably, the CSS design tokens include, but are not limited to, the following global CSS custom properties: --bg (primary background color), --bg-card (card background color), --bg-card-alt (card secondary background color), --ink (primary text color), --ink-soft (secondary text color), --ink-mute (weak text color), --primary (primary color tone), --primary-glow (primary color glow intensity), --accent (accordion color), --line (line color), --serif (serif font family), --sans (sans-serif font family), --mono (monospace font family).

[0019] When a user describes a custom style using natural language, the system calls a large language model to convert the natural language description into corresponding CSS design token values. For example, "dark blue tech style" is converted to --bg:#0A1422, --primary:#00D9FF, --accent:#FFB72B, and "Japanese minimalist style" is converted to --bg:#FAFAFA, --primary:#D4A5A5, --ink:#4A4A4A, etc. This achieves non-intrusive style switching by modifying only CSS variables without changing the HTML structure, whereas existing technologies usually require modifying the template file itself to achieve style customization.

[0020] Preferably, in steps S4 and S5, a three-layer audio-visual synchronization architecture is adopted: the logic layer defines the mapping relationship between the script and the animation through the beat sequence, the time layer calculates the global start time of each beat according to the TTS audio duration, and the execution layer precisely schedules the animation trigger, narration text and subtitle display in the browser according to the timeline through the controller, so as to achieve frame-level synchronization of the three.

[0021] Preferably, the synchronization accuracy of the three-layer audio-visual synchronization architecture is quantified as follows: After the TTS synthesis is completed, the time layer obtains the actual duration (not the estimated duration) of each audio segment through ffprobe, and calculates the global start time (startAt) one by one from the first segment to eliminate the cumulative error caused by the estimated duration of the text length; the execution layer checks the difference between the current playback time and the start time of the next beat in each frame on the browser side through the requestAnimationFrame API (approximately 60fps refresh rate). When the difference is less than 16ms (one frame duration), the corresponding animation is triggered immediately, ultimately achieving an audio-visual synchronization error of less than 33ms (exactly one frame when recording at 30fps). In terms of fault tolerance mechanism, the Conductor has built-in time calibration logic. After processing 5 beats (approximately 10-15 seconds), the current time offset is recalibrated based on performance.now to correct the cumulative time deviation caused by the browser event loop delay, ensuring the full-time synchronization stability of long videos of more than 30 minutes.

[0022] Preferably, in step S3, the layout density self-check is based on a multi-level canvas fill adaptive rule to ensure the screen is full from three levels: structure layer, volume layer, and density layer; by counting the number of cards, nodes, large number blocks, and list items, and combining the layout container type to estimate the number of vertical rows and pixel height, the risk of content overflow or blank space is detected.

[0023] Preferably, the specific execution logic of the multi-level canvas fill adaptive rule is as follows: The system takes the visible height of the stage (980px in the mode with subtitles and 1080px in the mode without subtitles) as the benchmark, and converts the number of elements such as cards, flow nodes, big-stat blocks, and list items in the HTML by counting the number of elements. Combined with the container type of grid layout (2 columns / 3 columns) and horizontal flow layout (flow-wave / flow-zigzag / flow-cross / flow-fishbone / flow-7), the number of elements is converted into an estimated number of vertical rows. The estimated total pixel height is obtained by multiplying the average height of each row (90px) by the number of rows and adding the top inner margin of 60px and the bottom inner margin of 100px. When the estimated height is less than 30% of the stage height, the system automatically adds fill elements to the volume layer, including converting the single-column grid layout to a two-column card grid (dilemma-grid), adding a big-stat-row, inserting decorative dividers (section-divider), and callout cards, etc.; when the estimated height exceeds 88% of the stage height, the system triggers a clipping warning, prompting the user to reduce the number of elements or adopt a more compact flow layout.

[0024] Preferably, it supports both fully automatic and manual review modes. In manual review mode, it pauses at three key points: after the outline is generated, it checks the number of slides and chapter segmentation; after the HTML and beat sequence are generated, it is interactively previewed and modified in the browser; and after the splicing is completed and finally confirmed, it enters the dubbing and screen recording stage.

[0025] Preferably, step S3 further includes a custom font embedding and SVG icon processing step: automatically scanning the user-provided custom font file and generating a @font-face declaration; embedding 22 geometric style SVG icons using sprite sheets and inheriting the theme color through stroke:currentColor.

[0026] Preferably, in step S2, the generation of the beat sequence follows the following constraints: the length of the explanatory text for each beat is 30-50 Chinese characters, each slide contains 4-12 beats, the same action is repeated no more than 3 times consecutively, each slide contains at least 3 different actions, and the first beat is a transition action.

[0027] Preferably, the technical basis for the above constraints is as follows: the standard Chinese teaching speed is approximately 250-280 characters per minute, and the explanation text of 30-50 Chinese characters corresponds to an audio duration of approximately 6-12 seconds, which conforms to the natural rhythmic unit of human auditory information reception—too short (less than 25 characters, corresponding to less than 5 seconds) results in insufficient information density, while too long (more than 60 characters, corresponding to more than 14 seconds) exceeds the capacity of auditory working memory. Each slide consists of 4-12 beats, with each beat lasting approximately 6-12 seconds, corresponding to an explanation time of approximately 30-90 seconds per slide, which conforms to the optimal dwell time for a single page of content in an instructional video—less than 30 seconds is insufficient to support the information on a single page, while more than 90 seconds can easily cause learners to lose focus. The constraints of not repeating the same action more than 3 times consecutively and each slide containing at least 3 different types of actions are based on the principle of "change blindness" in visual perception, aiming to avoid visual adaptation and viewing fatigue caused by monotonous animation modes and maintain learners' continuous attention.

[0028] Based on the above design, this application also designs an automated teaching video generation system based on document content, including: The document parsing module supports source document input in .docx / .md / .txt format and extracts structured text; The outline generation module is used to execute step S1 and output a structured outline file; The joint generation module is used to perform step S2 and output the HTML content and beat sequence of each slide; The splicing and verification module is used to perform step S3 and output a complete HTML page and a global beat sequence. The TTS dubbing and timeline module is used to perform step S4 and output a timeline file and synthesized audio. The browser screen recording module is used to execute step S5 and output the original video stream. The audio and video synthesis module is used to execute step S6 and output the final teaching video file; The scheduling module is used to schedule the execution flow of all the above modules with one click.

[0029] The advantages and effects of this application are as follows: 1. This application presents an automated method for generating instructional videos based on document content. It adopts a three-stage, four-step end-to-end automated pipeline technology solution, which solves the problems of fragmented production processes, high labor costs, and long production cycles in traditional teaching video production. It shortens the traditional production process of several days to several weeks to minutes (approximately 10-20 minutes for a 10,000-word document), significantly improving the production efficiency of teaching videos.

[0030] 2. This application presents an automated method for generating explanatory videos based on document content. It employs a technical solution that combines HTML and beat sequences for generation. By using a globally unique ID naming convention and a precise mapping mechanism for CSS selectors, it solves the common problem of missing or mismatched IDs during step-by-step generation, fundamentally ensuring the consistency between the animation target and HTML elements, and laying the foundation for audio-visual synchronization.

[0031] 3. This application presents an automated method for generating instructional videos based on document content. It employs a multi-style animation engine technology based on an action vocabulary, defining four categories of 15 standardized actions and 6 optional animation styles to form a 15×6 animation combination matrix. Driven by GSAP and incorporating random jitter to simulate natural hand-held rhythm, it solves the problems of insufficient animation expressiveness and monotonous style in existing technologies, significantly improving the visual effects and viewing experience of instructional videos.

[0032] 4. This application presents a method for automatically generating explanatory videos based on document content. It adopts a multi-style theme system and a user-defined template mechanism. It has three built-in preset visual themes, supports adaptive keyword matching and natural language custom styles, and reproduces the target style by modifying CSS design tokens. This solves the problem of fixed styles and lack of personalized customization in existing technologies, and meets the branding and personalization needs of different users.

[0033] 5. This application presents an automated method for generating explanatory videos based on document content. It adopts a three-layer audio-visual synchronization architecture driven by a beat. By defining the mapping relationship between the script and the animation in the logic layer, calculating the global start time in the time layer, and precisely scheduling the animation and subtitle updates in the execution layer, it solves the problem of audio-visual asynchrony in the prior art and achieves frame-level precise synchronization of animation triggering, narration text, and subtitle display.

[0034] 6. This application presents an automated method for generating instructional videos based on document content. It employs a multi-level canvas filling adaptive rule and an automated layout self-checking algorithm to ensure the fullness of the image from three levels: structure layer, volume layer, and density layer. By counting the number of elements and estimating the layout height, it detects the risk of overflow or blank space, thus solving the problem of insufficient or overflowing content in existing technologies and ensuring the aesthetics and readability of the instructional video.

[0035] 7. This application presents an automated method for generating explanatory videos based on document content. It employs a configurable manual review mechanism and supports both fully automated and manual review modes. The method pauses at three key nodes—outline generation, HTML and beat sequence generation, and splicing completion—to allow users to preview and modify the content. This solves the problems of existing technologies lacking flexible review and modification mechanisms and incurring high trial-and-error costs, thereby improving the controllability and accuracy of the generated results.

[0036] 8. This application presents an automated method for generating instructional videos based on document content. It employs a technical solution of custom font embedding and SVG icon system, automatically scans and generates @font-face declarations, incorporates 22 geometric style SVG icons, and inherits the theme color through stroke:currentColor. This solves the problem of inconsistent cross-platform rendering and ensures that the display effect of instructional videos is consistent across different devices.

[0037] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0038] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0040] Figure 1 This is an overall flowchart of the method for automatically generating teaching videos based on document content according to the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0042] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0043] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0044] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0045] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0046] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0047] Example 1, please refer to Figure 1 This embodiment mainly introduces a method and system for automatically generating instructional videos based on document content. The entire production process from source document to instructional video is divided into three core stages and four automated construction steps, forming a complete end-to-end pipeline. The entire pipeline is scheduled with a single click by build.py.

[0048] Three-stage content generation pipeline Phase 1A – Outline Generation The source document is structured and parsed to identify heading levels, paragraphs, lists, and other structures. The content is automatically divided into teaching slide units (usually 8-18 slides). Each unit is labeled with its content type (intro / definition / list / comparison / data / diagram / example / code / quote / summary, a total of 10 types), rhythm (fast / normal / slow), and visual layout suggestions, generating a structured outline file (outline.json).

[0049] Phase Two B – Joint Generation of HTML and Beat Sequence This is the core step of the invention. For each slide, HTML content (including layout, style, animable elements and their globally unique identifiers IDs following strict naming conventions) and beat sequences are generated simultaneously. Each beat defines the narration text, animation action, target element selector, and action parameters, achieving natural synchronization between animation and narration.

[0050] Phase 3 C—Assembly and Verification The HTML and beats of all slides are concatenated into a complete page, and automated validation (ID consistency check, text length validation, continuous action detection, layout density self-check, etc.) is performed to ensure the quality of the generated content.

[0051] Four-step automated build pipeline Step 1—TTS Dubbing and Timeline Injection It uses a cloud-based neural network speech synthesis engine (edge-tts, which supports 8 Chinese voice timbres) to synthesize audio by segmenting the text of each beat. It calculates the global start time (startAt) based on the actual duration of each audio segment, builds a timeline (timeline.json), and injects it into the HTML page.

[0052] Step 2—Automated Screen Recording via Browser Playwright is used to load HTML pages with timelines in a headless Chromium browser. The Conductor controller precisely drives page transitions and GSAP animations according to the timeline, while simultaneously recording 1920×1080 resolution and outputting a webm video stream.

[0053] Step 3—Audio and Video Synthesis Use ffmpeg to extract the video segment matching the audio duration from the end of the screen recording, transcode it into H.264 encoded MP4, merge the AAC audio, and output the final MP4 file (1920×1080, 30fps).

[0054] Detailed description of the seven core inventive points of this invention 1. Joint Generation Mechanism of HTML and Beat Sequence: The HTML content of each slide and the beat sequence are jointly generated in the same generation step. All animable elements in the HTML follow the globally unique ID naming convention s{slide number}-{element type}-{serial number / name}, specifically: titles use s{n}-title, subtitles use s{n}-subtitle, list items use s{n}-bullet-{i}, paragraphs use s{n}-text-{name}, keywords use s{n}-kw-{name}, cards use s{n}-card-{name}, steps use s{n}-step-{i}, and flow nodes use s{n}-node-{name}. Title elements (h1 / h2) do not have the anim-in class and are displayed when the page loads; elements targeted by the entry beat must have the anim-in class and are initially invisible. Each animation action in the beat sequence is precisely pointed to its corresponding ID in the HTML through a CSS selector. This mechanism fundamentally ensures the consistency between the animation target and the HTML element, eliminating the common ID missing or mismatch problems that occur during step-by-step generation.

[0055] 2. Multi-style animation engine based on an action vocabulary: Defines four categories and 15 standardized actions: transitions (page-in, section-change), entrances (text-in, bullet-in, card-in, image-in, diagram-step, split-reveal), emphasis (highlight, pulse, zoom-focus), and variation (number-counter, code-typewriter, hold). Each entrance action supports 6 selectable animation styles (float-up, wipe-right, wipe-left, scale-bounce, fade-only, pop-in), forming a 15×6 animation combination matrix, driven by GSAP (GreenSock Animation Platform), and incorporates random jitter (~8%) to simulate natural hand motion rhythm.

[0056] 3. Multi-style theme system and user-defined template mechanism: It includes three built-in preset visual themes (tech.css, research.css, and bioRender), each containing complete CSS design tokens and over 30 layout components. It supports three style selection paths: direct selection of preset themes; content-adaptive keyword matching (automatically selecting the corresponding style by scanning document theme words); and user-defined styles using natural language descriptions or specifying reference HTML templates. The system maintains the layout class name structure unchanged, only modifying the CSS design tokens (--bg, --bg-card, --ink, --primary, --accent, --serif, --sans, --mono, etc.) and shadow / rounded corner / glow parameters to reproduce the target style.

[0057] 4. Beat-driven precise audio-visual synchronization mechanism: Constructing a three-layer synchronization architecture: Logic layer (beats define the script → animation mapping) → Time layer (TTS audio duration calculation global startAt) → Execution layer (Conductor precisely schedules animation and subtitle updates in the browser according to the timeline). The three layers work together to ensure precise synchronization between animation triggering, narration text, and subtitle display.

[0058] 5. Multi-level Canvas Fill Adaptive Rules: Ensuring visual fullness through three layers: structural layer (each slide must contain at least three layers: title area + main area + bottom auxiliary area), volume layer (providing six filling methods including spread distribution, separator lines, large number rows, and catchy cards), and density layer (defining recommended rich layout schemes for 10 content types). This is complemented by an automated layout self-checking algorithm (counting the number of cards / nodes / lists, identifying layout container types, estimating the number of vertical rows and pixel height) to detect overflow or blank space risks. Specifically, regular expressions are used to count the number of cards (dilemma-card / compare-pane, etc.), nodes (.node), large number blocks (big-stat), and list items (...). The vertical row count and pixel height are estimated based on the layout container type (2-column grid / 3-column grid / horizontal flow / fishbone diagram). If the height exceeds 88% of the stage height, a warning for overflow is issued; if it is less than 30%, a warning for blank space is issued.

[0059] 6. Configurable manual review intervention mechanism: Supports both fully automatic and manual review modes. In manual review mode, pause at three key points: (1) After the outline is generated, check the number of slides and chapter division; (2) After HTML+beats is generated, perform an interactive preview in the browser (keyboard page turning, space bar to switch animation visibility), and you can modify the JSON data file and then run the preview again; (3) After the splicing is completed and finally confirmed, enter the dubbing screen recording.

[0060] 7. Custom Font Embedding and SVG Icon System: Supports users placing custom font files (.ttf / .otf / .woff / .woff2), automatically scanning and generating @font-face declarations during the splicing stage. Includes 22 built-in geometric style SVG icons (embedded in sprite format), inheriting theme colors via stroke:currentColor, replacing traditional emojis and ensuring consistent cross-platform rendering.

[0061] Complete processing flow First, users input the source document (supporting .docx / .md / .txt formats) and configure their preferences, including selecting the voiceover tone, theme style, and review mode.

[0062] Phase One: Outline Generation Phase The document parsing module extracts structural information such as heading levels, paragraphs, and lists from the source document; the outline generation module automatically breaks down the content into 8-18 logically coherent teaching slide units; the content annotation module annotates each slide unit with content type (10 types in total), rhythm (fast / normal / slow), and visual layout suggestions, and finally generates a structured outline file outline.json.

[0063] Phase Two B Joint Generation Phase (Core Steps): The joint generation module generates both HTML content and a beat sequence for each slide. The HTML content includes layout, styles, animable elements, and their globally unique IDs; the beat sequence includes narration text, animation actions, target element selectors, and action parameters. Beat sequence generation follows these constraints: the narration text for each beat is 30-50 Chinese characters long; each slide contains 4-12 beats; the same action is not repeated more than 3 times consecutively; each slide contains at least 3 different actions; and the first beat must be a page-in or section-change transition.

[0064] The third stage, C-level splicing and verification stage: The splicing and validation module splices the HTML content and beat sequences of all slides into complete slides.html and beats.json files; performs automated validations such as ID consistency checks, text length validation, continuous action detection, and layout density self-checks; and also completes custom font embedding and SVG icon processing.

[0065] Step 1: TTS Dubbing and Timeline Construction The TTS dubbing and timeline module uses the edge-tts engine to segment and synthesize audio clips from the narration text for each beat; it calculates the global start time startAt for each beat based on the actual duration of each audio clip, constructs the timeline.json timeline file, and then injects the timeline into the HTML page.

[0066] Step 2: Automated screen recording via browser: The browser screen recording module uses Playwright to launch a headless Chromium browser and load an HTML page containing a timeline. The Conductor works as follows: preloading (waiting for fonts and images to load, initializing anim-in elements to opacity:0, and displaying only the first slide) → launching (recording the launch timestamp, iterating through beats and waiting precisely by pressing startAt) → distributing (updating subtitle text → finding action handlers → building the execution context → calling the handler to execute the GSAP animation); it synchronously records 1920×1080 resolution, 30fps footage and outputs a raw.webm video stream.

[0067] Step 3: Audio and video synthesis: The audio and video synthesis module uses ffmpeg to extract segments from the end of the screen-recorded video stream that match the total audio duration; it then transcodes the video into H.264 encoded MP4, merges the AAC audio, and finally outputs the course.mp4 teaching video file (1920×1080, 30fps).

[0068] The present invention also discloses a system for implementing the above method, comprising: a document parsing module, an outline generation module, a joint generation module, a splicing and verification module, a TTS dubbing and timeline module, a browser screen recording module, an audio and video synthesis module, and a scheduling module for one-click scheduling of the execution flow of all modules.

[0069] The above description is merely a preferred embodiment of the present invention and does not limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter alterations to these embodiments within the spirit and principles of the present invention, achieved through conventional substitutions or by achieving the same function without departing from the principles and spirit of the present invention, fall within the scope of protection of the present invention.

Claims

1. A method for automatically generating explanatory videos based on document content, characterized in that, Includes the following steps: Step S1, Outline Generation Stage: Perform structured parsing on the input source document, identify the document's heading levels, paragraphs, and list structure, automatically divide the content into 8-18 teaching slide units, label each slide unit with content type, rhythm, and visual layout suggestions, and generate a structured outline file; Step S2, Joint Generation Stage: For each slide unit, simultaneously generate HTML content containing layout, style, animable elements and their globally unique identifier IDs, as well as a beat sequence containing explanatory text, animation actions, target element selectors and action parameters, to achieve joint output of HTML content and beat sequence; Step S3, splicing and verification stage: splice the HTML content and beat sequence of all slides into a complete page, and perform automated verification of ID consistency check, text length verification, continuous action detection, and layout density self-check; Step S4, TTS Dubbing and Timeline Construction: Using a cloud-based neural network speech synthesis engine, the narration text for each beat is segmented and synthesized into audio. The global start time is calculated based on the actual duration of each audio segment, and the timeline is constructed and injected into the HTML page. Step S5: Automated screen recording in a browser: Load an HTML page containing a timeline using a headless browser, and drive page switching and animation execution precisely according to the timeline using a controller, while simultaneously recording a 1920×1080 resolution screen output video stream. Step S6, Audio and Video Synthesis: Extract segments that match the audio duration from the screen recording video stream, transcode them to H.264 encoding format, merge the AAC audio, and output the final teaching video file.

2. The method for automatically generating explanatory videos based on document content according to claim 1, characterized in that, In step S2, all animable elements in the HTML follow the globally unique ID naming convention of s{slide number}-{element type}-{serial number / name}; each animation action in the beat sequence is precisely pointed to the corresponding ID in the HTML through a CSS selector, ensuring the consistency between the animation target and the HTML element.

3. The method for automatically generating explanatory videos based on document content according to claim 1, characterized in that, In step S5, the animation execution is based on a multi-style animation engine. The multi-style animation engine defines 15 standardized actions in four categories: transition, entrance, emphasis, and variation. Each entrance action supports 6 selectable animation styles, forming a 15×6 animation combination matrix. It is driven by GSAP and incorporates up to 8% random jitter to simulate the rhythm of natural hand movements.

4. The method for automatically generating explanatory videos based on document content according to claim 1, characterized in that, Before step S1, there is also a user preference configuration step, which allows users to select a preset visual theme, adaptively match the style by document theme words, or customize the style by describing it in natural language and specify a reference HTML template; the system keeps the layout class name structure unchanged and only modifies the CSS design token to reproduce the target style.

5. The method for automatically generating explanatory videos based on document content according to claim 1, characterized in that, In steps S4 and S5, a three-layer audio-visual synchronization architecture is adopted: the logic layer defines the mapping relationship between the script and the animation through the beat sequence, the time layer calculates the global start time of each beat according to the TTS audio duration, and the execution layer precisely schedules the animation trigger, narration text and subtitle display in the browser according to the timeline through the controller, so as to achieve frame-level synchronization of the three.

6. The method for automatically generating explanatory videos based on document content according to claim 1, characterized in that, In step S3, the layout density self-check is based on multi-level canvas fill adaptive rules to ensure the screen is full from three levels: structure layer, volume layer, and density layer. By counting the number of cards, nodes, large number blocks, and list items, and combining the layout container type, the vertical row number and pixel height are estimated to detect the risk of content overflow or blank space.

7. The method for automatically generating explanatory videos based on document content according to claim 1, characterized in that, It supports both fully automatic and manual review modes. In manual review mode, it pauses at three key points: after the outline is generated, it checks the number of slides and chapter segmentation; after the HTML and beat sequence are generated, it allows interactive preview and modification in the browser; and after the splicing is completed and finally confirmed, it enters the dubbing and screen recording stage.

8. The method for automatically generating explanatory videos based on document content according to claim 1, characterized in that, Step S3 also includes custom font embedding and SVG icon processing steps: automatically scanning user-provided custom font files and generating @font-face declarations; embedding 22 geometric style SVG icons using sprite sheets and inheriting the theme color through stroke:currentColor.

9. The method for automatically generating explanatory videos based on document content according to claim 1, characterized in that, In step S2, the generation of the beat sequence follows the following constraints: the length of the explanatory text for each beat is 30-50 Chinese characters, each slide contains 4-12 beats, the same action is repeated no more than 3 times consecutively, each slide contains at least 3 different actions, and the first beat is a transition action.

10. A system for automatically generating instructional videos based on document content, the system being used to implement the method for automatically generating instructional videos based on document content as described in any one of claims 1-9, characterized in that, include: The document parsing module supports source document input in .docx / .md / .txt format and extracts structured text; The outline generation module is used to execute step S1 and output a structured outline file; The joint generation module is used to perform step S2 and output the HTML content and beat sequence of each slide; The splicing and verification module is used to perform step S3 and output a complete HTML page and a global beat sequence. The TTS dubbing and timeline module is used to perform step S4 and output a timeline file and synthesized audio. The browser screen recording module is used to execute step S5 and output the original video stream. The audio and video synthesis module is used to execute step S6 and output the final teaching video file; The scheduling module is used to schedule the execution flow of all the above modules with one click.

Images