A Method and System for Automatic Generation of Micro-lesson Videos Based on Anchor Point Event Graphs

By constructing semantic unit graphs and anchor event graphs, the problems of poor content traceability and low multi-track synchronization accuracy in micro-lesson video generation are solved, achieving accurate mapping between explanation content and materials and timeline synchronization, thus improving update efficiency.

CN123123667APending Publication Date: 2026-11-17YIYUNYING NETWORK TECHNOLOGY (JINAN) CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202611234929.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-14
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing micro-lesson video generation technologies suffer from poor content traceability, low multi-track synchronization accuracy, and low update efficiency. In particular, when materials change, it is impossible to accurately locate and update the affected segments.

Method used

By constructing semantic unit graphs and anchor event graphs, the input courseware is parsed into page element trees and visual region trees, generating explanation segments and binding semantic unit identifiers. The unified modeling is an anchor event graph with phoneme boundaries as the main anchor points for local time alignment, achieving precise synchronization of various media tracks.

Benefits of technology

It achieves a precise mapping between the explanation content and the original materials, solving the problems of missing explanation content, disordered order, and time drift, and transforms it into a traceable and verifiable automated generation link, thus improving update efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123667A_ABST
    Figure CN123123667A_ABST
Patent Text Reader

Abstract

This invention discloses a method and system for automatically generating micro-lesson videos based on anchor point event graphs. The method includes: acquiring input courseware; constructing a semantic unit graph based on the input courseware, with logical order between semantic units; generating a sequence of explanation segments based on the semantic unit graph, and binding a semantic unit identifier to each explanation segment; generating speech data and phoneme time stamps, as well as courseware switching, subtitle, and dynamic annotation events; driving a digital human to generate video tracks; uniformly constructing each media track into an anchor point event graph, where each anchor point includes event type, start and end time, and priority, and establishing timing constraints according to the playback order; performing local time alignment based on phoneme boundaries, and outputting a synthesized timeline to generate a micro-lesson video. This invention achieves traceability of explanation content and precise synchronization across multiple tracks, improving the generation quality and update efficiency of micro-lesson videos.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of multimedia information processing technology, specifically relating to a method and system for automatically generating micro-lesson videos based on anchor point event graphs. Background Technology

[0002] With the rapid development of online education and corporate training, the demand for micro-lecture video production is increasing. Traditional micro-lecture production methods typically require instructors to record live in a studio or recording classroom, followed by post-production editing, subtitle proofreading, and special effects addition. This method is not only time-consuming and costly, but also necessitates the production team to re-record complete video courses whenever policies, product parameters, or operating procedures change, making it difficult to adapt to the high-frequency updates required for corporate training content.

[0003] In recent years, some technical solutions have attempted to automatically generate micro-lecture videos using text generation models and digital human technology. These solutions typically input the full text of documents such as PPT, PDF, or Word documents into a text generation model, which then generates a script. This script is then fed into a speech synthesis engine to generate audio, and a digital human platform is used to generate lip-sync animation and body movements. However, this approach has the following drawbacks: First, there is a lack of traceable mapping between the script generated by the text generation model and the pages, paragraphs, and charts of the original document. This can easily lead to omissions of important knowledge points, incorrect presentation order, or the generation of content beyond the scope of the input material. Business personnel find it difficult to verify the accuracy and completeness of the presentation content. Second, speech synthesis, digital human animation, slide flipping (slide switching), subtitle generation, and dynamic annotation are usually generated by different independent engines. Each engine uses different time granularities and output durations, which can easily cause timeline drift problems during synthesis, such as lip-syncing delayed by speech, slide flipping preceding the presentation, misaligned subtitles across sentences, and key annotations pointing to the wrong areas. Existing solutions typically only stretch or compress each media track as a whole based on its total duration, failing to handle local time deviations at the sentence or even phoneme level. Third, when local changes occur to the input material, due to the lack of fine-grained management of content dependencies, the system often needs to regenerate the complete video file, failing to accurately locate the affected segments and update only those segments.

[0004] Therefore, there is an urgent need for a method that can convert input materials into a structured representation with source anchors and time constraints, and on this basis, complete multi-track timeline alignment through a unified anchor event graph, and support the automatic generation of micro-lesson videos that only regenerates the affected segments when there are local changes in the materials, so as to solve the problems of poor content traceability, low multi-track synchronization accuracy and low update efficiency in the existing technology.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the embodiments of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0007] This disclosure provides a method and system for automatically generating micro-lesson videos based on anchor point event graphs. It parses input courseware into a page element tree and a visual region tree, constructing a semantic unit graph to give each taught knowledge point a clear source anchor point. Based on this, it uniformly models speech, phonemes, explanation segments, courseware switching, subtitles, digital human actions, and dynamic annotations into an anchor point event graph, using phoneme boundaries as the primary anchor point for local time alignment. The semantic unit graph provides traceable material constraints for script generation, establishing a precise mapping relationship between the explanation content and the original materials, fundamentally solving problems such as missing content, disordered order, and content exceeding the material's scope. The anchor point event graph provides a unified spatiotemporal modeling framework for multi-track alignment, ensuring precise synchronization of each media track on the playback timeline, fundamentally overcoming time drift problems such as lip movements later than speech, page turns earlier than explanation, and subtitles spanning sentences. Through the synergy of these mechanisms, the micro-lesson production process, which originally required significant manual intervention, is transformed into a traceable and verifiable automated generation chain, effectively solving the problems of poor content traceability and low multi-track synchronization accuracy in existing technologies.

[0008] In some embodiments, a method for automatically generating micro-lesson videos based on anchor point event graphs is provided, including the following steps:

[0009] Obtain input courseware, construct a semantic unit graph based on the input courseware, wherein the semantic units in the semantic unit graph have a logical order, and each semantic unit is associated with a teaching knowledge point in the input courseware;

[0010] A sequence of explanatory segments is generated based on the logical order between semantic units, and at least one semantic unit identifier is bound to each explanatory segment in the sequence of explanatory segments.

[0011] Based on the sequence of explanatory segments, speech data and corresponding phoneme time markers, word boundary time markers, and prosodic markers are generated, as well as courseware switching events, subtitle events, and dynamic annotation events.

[0012] The digital human is driven to generate a digital human video track containing lip-sync sequences and body movements based on the phoneme time markers and the prosodic markers.

[0013] Construct an anchor event graph, wherein each anchor point in the anchor event graph contains at least an event type, start and end time, and priority, and establish a sequential time constraint relationship between each anchor point according to the playback order of the explanation segment sequence;

[0014] Based on the sequential temporal constraints of the anchor event graph and the priority of each anchor point, the events corresponding to each anchor point are locally time-aligned using the phoneme boundaries in the speech data as the time reference, and a synthesized time axis is output.

[0015] The micro-lesson video is output based on the synthesized timeline.

[0016] In some embodiments, a method system for automatically generating micro-lesson videos based on anchor event graphs is provided, including:

[0017] The semantic unit graph construction module acquires input courseware and constructs a semantic unit graph based on the input courseware. The semantic units in the semantic unit graph have a logical order, and each semantic unit is associated with a teaching knowledge point in the input courseware.

[0018] The explanation segment sequence generation module generates an explanation segment sequence according to the logical order between semantic units, and binds at least one semantic unit identifier to each explanation segment in the explanation segment sequence.

[0019] The data generation module is used to generate speech data and corresponding phoneme time markers, word boundary time markers and prosodic markers according to the sequence of the explanation segments, as well as to generate courseware switching events, subtitle events and dynamic annotation events;

[0020] The digital human driving module is used to drive the digital human to generate a digital human picture track containing lip-sync sequences and body movements based on the phoneme time markers and prosodic markers.

[0021] The timeline alignment module is used to construct an anchor event graph based on the audio data, the sequence of explanatory segments, courseware switching events, subtitle events, body movement segments in the digital human video track, and dynamic annotation events. Each anchor point in the anchor event graph contains at least event type, start and end time, and priority attributes, and a sequential temporal constraint relationship is established between each anchor point according to the playback order of the explanatory segment sequence. The module is also used to perform local time alignment on the corresponding courseware switching events, subtitle events, body movement segments in the digital human video track, and dynamic annotation events, using the phoneme boundaries in the audio data as the time reference, based on the sequential temporal constraint relationship of the anchor event graph and the priority of each anchor point, and output a synthesized timeline.

[0022] The video output module is used to output micro-lesson videos according to the synthesized timeline.

[0023] In some embodiments, an electronic device is provided, comprising: at least one processor; and

[0024] A memory that is communicatively connected to the at least one processor;

[0025] The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the processor to perform the micro-lesson video automatic generation method based on anchor event graph as described in the first aspect.

[0026] The micro-lesson video automatic generation method, system, and electronic device based on anchor point event graphs provided in this disclosure can achieve the following technical effects:

[0027] This disclosure provides a method and system for automatically generating micro-lesson videos based on anchor point event graphs. By constructing semantic unit graphs, it provides traceable material constraints for script generation. Each knowledge point is bound to its source page number, original text range, concept tag, and visual region coordinates, establishing a precise mapping relationship between the content and the original materials. This fundamentally solves the problems of missing content, disordered order, and content exceeding the material's scope. The anchor point event graph provides a unified spatiotemporal modeling framework for multi-track alignment. It uniformly models speech, phonemes, explanation segments, courseware switching, subtitles, digital human actions, and dynamic annotations as anchor point event graphs with priority and temporal constraints. Local time alignment is performed using phoneme boundaries as primary anchor points, ensuring precise synchronization of each media track on the playback timeline. This fundamentally overcomes time drift problems such as lip movements later than speech, page turns earlier than explanation, and subtitles spanning sentences. Together, they transform the micro-lesson production process, which originally required significant manual intervention, into a traceable and verifiable automated generation chain, effectively solving the technical problems of poor content traceability and low multi-track synchronization accuracy in existing technologies.

[0028] The above general description and the description below are exemplary and explanatory only, and are not intended to limit the embodiments of this disclosure. Attached Figure Description

[0029] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein:

[0030] Figure 1 This is a schematic diagram of the structure of the micro-lesson video automatic generation method system based on anchor point event graph provided in this embodiment of the disclosure;

[0031] Figure 2 This is a flowchart illustrating the method for automatically generating micro-lesson videos based on anchor point event graphs provided in this embodiment of the disclosure.

[0032] Figure 3 This is a flowchart illustrating the construction process of the semantic unit graph in this embodiment of the disclosure;

[0033] Figure 4 This is a flowchart illustrating the construction process of the anchor event graph in this embodiment of the disclosure;

[0034] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this disclosure. Detailed Implementation

[0035] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0036] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0037] Unless otherwise stated, the term "multiple" means two or more.

[0038] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0039] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0040] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0041] Figure 1 This disclosure presents a method and system for automatically generating micro-lesson videos based on anchor event graphs. This system is deployed on a server cluster of an online learning platform or enterprise training system. (Refer to...) Figure 1 As shown, the system includes: a semantic unit graph construction module, a narration segment sequence generation module, a data generation module, a digital human driving module, a timeline alignment module, and a video output module.

[0042] The semantic unit graph construction module constructs a semantic unit graph based on the acquired input courseware. The semantic units in the semantic unit graph have a logical order, and each semantic unit is associated with a teaching knowledge point in the input courseware.

[0043] In one embodiment, the input courseware is parsed into a page element tree and a visual region tree, and then a semantic unit graph is constructed based on these two trees. The page element tree records the hierarchical structure of the document, including which page elements are present and their parent-child relationships. The visual region tree records the spatial coordinates of each page element within the page. The visual region tree and the page element tree share the same node identifiers, enabling a correspondence between the logical position (position in the hierarchical structure) and the physical position (visual range within the page) of each teachable element.

[0044] The page element tree is traversed, and several page elements belonging to the same teaching unit are combined into a semantic unit. During combination, the visual range of each page element is read from the visual region tree and merged as the visual region coordinates of the semantic unit. After combination, directed edges are established between semantic units to form a semantic unit graph. The establishment of directed edges is based on title inclusion relationships, page adjacency relationships, referential relationships, and concept sequence relationships. These directed edges record the logical order between various knowledge points. Each semantic unit contains at least the source page number, original text range, concept tag, and visual region coordinates of the knowledge point in the input courseware, providing a traceability basis for the generation of subsequent lecture notes and the positioning of dynamic annotations.

[0045] The explanatory segment sequence generation module generates a sequence of explanatory segments according to the logical order between semantic units, and binds at least one semantic unit identifier to each explanatory segment. The generation of explanatory segments can employ text generation techniques based on large language models. In one specific implementation, to ensure that the generated explanatory text does not exceed the scope of the input material, the system adopts an "extract-generate" framework. First, the system assembles the original text scope, concept labels, and visual region coordinate information from the semantic units into structured prompts, which are then input into a pre-trained large language model (such as commercial models like GPT-4, Wenxin Yiyan, or Tongyi Qianwen, or open-source models like LLaMA). These prompts explicitly require the model to perform the task of "rewriting the explanatory text based on the given material," and provide examples (few-shots). For example, given the original text "A leads to B," the corresponding explanatory text would be "A is one of the main reasons for B." Through this example constraint, the model is guided to understand and paraphrase the material. During the generation of each explanatory segment, the system simultaneously records the semantic unit identifiers it references, thereby establishing a binding relationship between the explanatory content and the source of the knowledge points.

[0046] The data generation module generates speech data and corresponding phoneme time stamps, word boundary time stamps, and prosodic markers based on the sequence of explanatory segments, as well as generating courseware switching events, subtitle events, and dynamic annotation events. Speech data is generated by the speech synthesis engine from the explanatory text. Phoneme time stamps record the start and end times of each phoneme in the audio, word boundary time stamps record the start and end times of each word, and prosodic markers record information such as stress, pauses, and intonation boundaries. Courseware switching events are generated based on the source page number in the semantic unit bound to the explanatory segment. When an explanatory segment begins explaining the content of a certain page, the system generates a courseware switching event, indicating that the courseware screen should switch to the corresponding page at that moment. Dynamic annotation events are generated based on the source page number, visual region coordinates, and concept tags in the semantic unit bound to the explanatory segment. The system uses several key elements: the source page number to determine the target page where the annotation is located; the visual region coordinates to determine the specific display position of the annotation on that target page (e.g., the coordinates of the top left and bottom right corners of a rectangle, or the position of an arrow); the concept tag to identify keywords within the knowledge point; and the data generation module to search for words matching the keywords in the word boundary time stamps corresponding to the current explanation segment. If a unique word is matched, its start time is used as the start time and end time as the end time of the dynamic annotation. If multiple words are matched, the system selects one according to a preset strategy: for example, selecting the word that first appears on the timeline, or selecting the word corresponding to the page element whose visual region coordinates are spatially closest to the current explanation segment (determined by calculating the distance between the word's position in the original text and its visual region coordinates). This matching strategy achieves a precise synchronization effect where the annotation appears when the keyword begins to be pronounced and disappears when the keyword ends to be pronounced. The source page number, visual region coordinates, and the matched display interval together determine the spatial position and display / disappearance time of the dynamic annotation.

[0047] The digital human driving module is used to drive the digital human to generate a video track containing lip-sync sequences and body movements based on phoneme time markers and prosodic markers. The digital human video track is a video frame with a transparent background so that it can be overlaid and composited with the courseware footage later.

[0048] The timeline alignment module constructs an anchor event graph based on audio data, lecture segment sequences, courseware switching events, subtitle events, body movement segments in the digital human video track, and dynamic annotation events. The anchor event graph is organized using a directed acyclic graph (DAG) data structure, where each anchor point is a node, and the temporal constraints between anchor points are directed edges. Each node records at least the following fields: event type, start and end times, and priority; it may also include the media data index identifier corresponding to that anchor point. The anchor event graph can be stored in an in-memory graph database (such as Neo4j), in a node and edge table of a relational database, or persistently stored as a structured file in JSON / XML format. The timeline alignment module also performs local time alignment of the corresponding courseware switching events, subtitle events, body movement segments in the digital human video track, and dynamic annotation events, based on the temporal constraints of the anchor event graph and the priority of each anchor point, using the phoneme boundaries in the audio data as the time reference, and outputs a synthesized timeline.

[0049] The video output module is used to render and synthesize the digital human video track, courseware video track, subtitle track, and dynamic annotation track into a micro-lesson video according to the synthesis timeline.

[0050] The micro-lesson video automatic generation system based on anchor point event graphs disclosed in this invention generates a semantic unit graph by structuring the input courseware through a semantic unit graph construction module. A lecture segment sequence generation module generates corresponding lecture segment sequences according to the logical order of each semantic unit. These lecture segment sequences are then passed to a data generation module and a digital human-driven module. The media tracks output by the data generation and digital human-driven modules are aggregated by a timeline alignment module for unified modeling and alignment. Finally, the video output module outputs the micro-lesson video. All modules communicate through standardized data interfaces, forming a complete processing chain from material input to video output.

[0051] Example 2

[0052] Corresponding to Embodiment 1 above, Embodiment 2 of the present invention provides a method for automatically generating micro-lesson videos based on anchor point event graphs, referring to... Figure 2 As shown, the method includes the following steps:

[0053] S1: Obtain the input courseware, and construct a semantic unit graph based on the input courseware. The semantic units in the semantic unit graph have a logical order, and each semantic unit is associated with a teaching knowledge point in the input courseware.

[0054] Reference Figure 3 As shown, the construction process of the semantic unit graph includes:

[0055] S11. Parse the input courseware into a page element tree and a visual region tree.

[0056] User-uploaded input courseware can be in PPT, PDF, or Word format. For PPT, the system reads slide objects, speaker notes, animation order, and page size; for PDF, it retrieves text positions, vector objects, and page images; and for Word, it extracts heading levels, paragraphs, tables, images, and pagination relationships. Then, two types of structured information are extracted from the parsed results: the first is a page element tree, a tree-like data model organized according to the document's hierarchical structure. Each node in the tree corresponds to a page element, and the parent-child relationships between nodes record the hierarchical relationships between page elements. A page element refers to every independently identifiable and locatable content unit in the input courseware. For PPT, this includes title text boxes, body text boxes, shapes, images, charts, tables, and notes; for PDF, it includes text blocks, vector graphics, and embedded images; and for Word, it includes paragraphs, tables, images, and charts. For example, a slide node contains all element nodes for that page, a Word document node contains chapter nodes, chapter nodes contain section nodes, and section nodes contain paragraph nodes. The second type is the visual region tree, which records the spatial coordinates of each page element within the page to represent the visual range, i.e., the rectangular area occupied by the element on the page, represented by the coordinates of the top-left and bottom-right corners of that rectangular area. For example, the visual range of a title text box might be "top-left (50, 30), bottom-right (800, 80)", and the visual range of a chart might be "top-left (100, 200), bottom-right (700, 600)". The visual region tree shares the same node identifiers as the page element tree, enabling the logical position (position in the hierarchical structure) of each teachable element to correspond to its physical position (visual range on the page). The page element tree provides the logical structure of the content, while the visual region tree provides the spatial distribution of the content; together, they form the foundation for the subsequent construction of semantic units.

[0057] S12. Construct a semantic unit diagram based on the page elements in the page element tree and the visual regions in the visual region tree.

[0058] The semantic unit graph contains multiple semantic units and directed edges between them. Each semantic unit represents an independent teaching knowledge point, which is the smallest content aggregation unit in the courseware with independent teaching semantics, serving as the basic information carrier for a complete explanation. A teaching knowledge point contains at least one core theme and its directly related explanatory content. When multiple page elements jointly explain the same core theme and have semantic relationships such as subordinate titles, explanatory text, diagram descriptions, or supplementary notes, these page elements are grouped into semantic units corresponding to the same teaching knowledge point. The specific construction process is as follows:

[0059] First, the page element tree is traversed, and several page elements belonging to the same teaching unit are grouped into a semantic unit. This means that the content of these page elements is aggregated into a knowledge point unit. For example, a title "Accident Classification and Grading" and its three list items "General Accidents," "Major Accidents," and "Serious Accidents" are originally four independent page elements, but they constitute a complete teaching knowledge point (accident classification and grading) and are therefore grouped into one semantic unit. The criteria for determining "which page elements belong to the same teaching unit" are as follows: For PPT format, the title and its subordinate content (paragraphs, list items, charts and their descriptions) that constitute a complete teaching unit on each slide are grouped into a semantic unit; for PDF or Word format, the text content under the same title is grouped into a semantic unit according to the title hierarchy. Purely decorative page elements (such as background images and separators) are excluded during the grouping process because they do not contain substantive teaching content.

[0060] After combination, each semantic unit contains at least the following information: source page number (taken from the page where its contained page elements are located), original text range (taken from the original text content of its contained page elements), concept tags (generated by the system based on keyword extraction algorithms that extract keywords according to the titles and core content covered by the taught knowledge points, including but not limited to statistical extraction methods based on term frequency-inverse document frequency (TF-IDF) and graph model extraction methods based on word co-occurrence graph ranking (TextRank), with the top-ranked words in the extraction results serving as the concept tags of the semantic unit), and visual region coordinates, which are calculated by merging the visual ranges of the page elements it contains. The visual ranges of each page element are summarized, and the bounding rectangle of the whole is taken as the visual region coordinates of the semantic unit, which can be represented by absolute pixel coordinates or page percentage coordinates.

[0061] Then, directed edges are established between semantic units to form a semantic unit graph. These directed edges are established based on one or more of the following relationships: title inclusion relationship, meaning the semantic unit corresponding to the parent title should be explained before the semantic unit corresponding to the child title (e.g., "Accident Classification and Grading" includes the three sub-concepts "General Accident," "Major Accident," and "Serious Accident"); page adjacency relationship, meaning knowledge blocks spatially adjacent on the same page are explained sequentially according to reading order (e.g., adjacent knowledge blocks on a page); referential relationship, meaning the semantic unit corresponding to the referred to chart or concept should be explained before the semantic unit containing the referent (e.g., "as shown in the above figure" in the text points to the corresponding chart semantic unit); and concept sequence relationship, meaning a preceding concept with a causal, progressive, or hierarchical relationship should be explained before the following concept (e.g., "Introduce the concept first, then the classification"). Through these directed edges, the semantic unit graph records the logical order between the knowledge points, providing a sequential basis for the generation of subsequent explanation segments.

[0062] The semantic unit diagram constructed through step S1 above provides each knowledge point with a clear source anchor: which page it comes from, which paragraph of the original text it corresponds to, which area of ​​the page it is located in, and which core concepts it involves. This information provides a precise basis for the generation of the lecture script, the control of slide turning (slide switching), and the positioning of dynamic annotations in subsequent steps.

[0063] S2: Generate a sequence of explanatory segments based on the logical order between semantic units, and bind at least one semantic unit identifier to each explanatory segment.

[0064] Specifically, the explanation segment sequence generation module traverses the semantic unit graph sequentially according to the logical order determined by the directed edges between nodes (i.e., the order in which knowledge points are taught), generating corresponding explanation text for each semantic unit. These explanation texts, arranged in sequence, form the explanation segment sequence. The generation of explanation segments can employ text generation techniques based on large language models: using the original text scope, concept tags, and visual region information within the semantic unit as input prompts, the model is required to generate explanation text for that knowledge point. During the generation process, the model is constrained to explain only based on the input material content and cannot arbitrarily add extended content beyond the scope of the material. In most cases, one semantic unit corresponds to one explanation segment, forming a one-to-one relationship. However, this solution does not require a strict fixed mapping. When a semantic unit contains rich content (e.g., a title contains multiple sub-concepts, or a complex chart needs to be interpreted step by step, or the original text it contains exceeds the preset word count threshold), the system can split it into multiple explanatory segments and explain them separately, which is a one-to-many relationship. When several consecutive semantic units are short (e.g., all below the preset word count threshold) and highly related, the system can also merge them into one explanatory segment and explain them together, which is a many-to-one relationship.

[0065] When each explanatory segment is generated, the system simultaneously records one or more semantic unit identifiers that it is bound to. In a one-to-many scenario, the multiple explanatory segments are bound to the same semantic unit identifier; in a many-to-one scenario, the merged explanatory segment is bound to multiple semantic unit identifiers. Through this binding relationship, each explanatory content can be traced back to one or more knowledge points. This binding relationship establishes a precise mapping relationship between the explanatory content and the original materials.

[0066] During the process of generating the explanation script, in order to ensure that the explanation content fully covers all knowledge points and does not exceed the scope of the materials, after generating the explanation segment sequence, the explanation segment sequence generation module also calculates the key concept coverage rate of the semantic units referenced by each explanation segment relative to all effective semantic units participating in the explanation in the semantic unit graph, as well as the source support rate of the number of sentences in each explanation segment whose content is supported by the original text scope of the semantic units bound to it relative to the total number of sentences in the explanation segment.

[0067] Key concept coverage refers to the proportion of semantic units referenced by at least one explanatory segment out of all valid semantic units (excluding purely decorative page elements). For example, if there are 10 valid semantic units in the semantic unit diagram (excluding decorative and low-priority elements), and the generated explanatory segment sequence only references 8 of them, then the key concept coverage is 80%. The explanatory segment sequence generation module compares the calculated coverage with a preset first threshold (e.g., 95%). If the coverage is lower than the first threshold, it indicates that some knowledge points have not been explained. The module marks the semantic units not referenced by any explanatory segment as uncovered semantic units and triggers regeneration, generating new explanatory segments only for these uncovered semantic units, rather than regenerating the entire explanatory script.

[0068] Source support rate refers to the proportion of sentences in each explanatory segment whose content is supported by the original text range of the bound semantic units, out of the total number of sentences in that explanatory segment. For example, if an explanatory segment has 10 sentences, and the content of 8 of them can be directly found in the original text range of their bound semantic units, while the other 2 sentences are extended content added by the generation model, then the source support rate is 80%. The explanatory segment sequence generation module compares the calculated source support rate with a preset second threshold (e.g., 90%). If the source support rate is lower than the second threshold, it indicates that there is a lot of "free-flowing" content in the explanatory segment without source support. The module marks sentences lacking original text range support as sourceless sentences and regenerates explanatory segments containing these sentences. During regeneration, the model is forced to base its expression on the original text range.

[0069] The calculation and regeneration processes for coverage and source support are performed cyclically until both reach their respective thresholds. This dual verification mechanism ensures that the explanation script fully covers all knowledge points (high coverage) while remaining faithful to the original material (high support), avoiding the problems of omissions or arbitrary expansions without source material support found in existing technologies. After the cyclical verification process is completed, the system records the identifiers of all explanation segments marked as to be rewritten and already regenerated. In the subsequent S3 step, the system only performs speech data generation and extracts phoneme time stamps, word boundary time stamps, and prosodic markers for these regenerated explanation segments in the record; for explanation segments that have passed verification and have not changed, the system reuses the speech data and time stamps saved during initial generation to avoid redundant calculations.

[0070] S3: Generate speech data and corresponding phoneme time markers, word boundary time markers and prosodic markers according to the sequence of the explanatory segments, as well as generate courseware switching events, subtitle events and dynamic annotation events.

[0071] The data generation module receives the sequence of narration segments and their rewrite markers output from step S2. For narration segments not marked as rewrites, the system reuses the audio data and time stamps saved during the initial generation; for narration segments marked as rewrites, the corresponding audio data and time stamps are regenerated according to the complete process of this step, and the system records are updated. Specifically, the data generation module performs the following two types of generation tasks in parallel based on the sequence of narration segments generated in step S2:

[0072] The first type of task is the generation of speech data and time stamps. The data generation module sends the narration segments marked as needing generation (i.e., initial generation or marked as rewriting) from the narration segment sequence to the speech synthesis engine segment by segment. The speech synthesis engine outputs the corresponding audio waveform data independently for each narration segment, and returns the phoneme time stamps, word boundary time stamps, and prosodic stamps for that audio. The audio is divided into three parts: Phoneme time markers, which are lists of phoneme symbols and their start and end times in the audio, recording the start and end times of each phoneme (such as basic articulation units like / k / , / æ / , / t / in English, or initials and finals in Chinese), with millisecond precision; and word boundary time markers, which are lists of word text and their start and end times, recording the start and end times of each word. Prosodic markers include at least pause markers, stress markers, and intonation markers, used to indicate the degree of stress on each syllable in the explanatory text, the pause positions in sentences, and the intonation direction at the end of sentences. For example, when explaining the phrase "safety production management," the speech synthesis engine will mark the natural pause position between "safety" and "production management," as well as the stress position on the word "management." These prosodic markers are subsequently used to drive the rhythm of body movements and facial expressions in the digital human's visual track. It should be noted that the phoneme time markers and prosodic markers generated in this step are mainly used to drive the digital human's lip movements and basic movement rhythms. Among them, prosodic markers contain information such as stress, pauses, and intonation boundaries. This information is not only used for speech synthesis, but also serves as prosodic markers for the speech layer.

[0073] The second type of task involves generating courseware switching events, subtitle events, and dynamic annotation events. The generation rules for courseware switching events are as follows: The data generation module obtains the source page number from the semantic unit bound to each explanation segment. For the i-th explanation segment in the sequence, if its bound source page number is different from the source page number bound to the (i-1)-th explanation segment, a courseware switching event is generated at the start time T_i of that explanation segment. This event indicates that the courseware screen should be switched to the page corresponding to that source page number at time T_i. If the same source page number is referenced by multiple consecutive explanation segments, a switching event is only generated at the start time of the first explanation segment that references that page number; subsequent explanation segments maintain the current courseware page unchanged. The rules for generating subtitle events are as follows: The data generation module obtains the text content of the explanation segment and its word boundary time markers, and divides the text into complete sentences according to periods, question marks, and exclamation marks as sentence boundaries; for sentences whose length exceeds the upper limit of characters per line (preset to no more than 20 Chinese characters or equal-width English words per line), it further splits them at secondary syntactic boundaries such as commas, semicolons, and colons to ensure that the number of characters in each subtitle does not exceed the upper limit. The display interval of each subtitle is bound to the start time of the first word and the end time of the last word in all the words contained in the subtitle. The rules for generating dynamic annotation events are as follows: Dynamic annotation events are generated based on the source page number, visual area coordinates, and concept tags in the semantic unit bound to the explanation segment. As a data object, a dynamic annotation event contains at least the following fields: target page number, visual area coordinates, concept keywords, display start time, display end time, and annotation type (highlighted box, underline, or arrow).

[0074] The source page number is used to determine the target courseware page where the annotation is located; the visual area coordinates are used to determine the specific display position of the annotation on the target page (e.g., the coordinates of the top left and bottom right corners of a rectangle, or the position pointed to by an arrow). These coordinates are stored in a page percentage coordinate format, that is, the position of the rectangular area is represented as a percentage of the page width and height (e.g., the top left corner coordinates are (10%, 20%), and the bottom right corner coordinates are (30%, 25%)), making the coordinate position independent of the output resolution; the concept tag is used to identify the keywords in the knowledge point. The data generation module searches for words matching the keywords in the word boundary time stamps, and uses the start time of the word as the display start time of the dynamic annotation, and the end time as the display end time, thereby achieving a precise synchronization effect where the annotation appears when the keyword begins to be pronounced and disappears when the keyword ends to be pronounced. The three together determine the spatial position and display time of the dynamic annotation.

[0075] It's important to note that subtitles display the explanatory text sentence by sentence at the bottom of the screen, with their display area typically corresponding to the playback interval of a complete sentence or segment. Dynamic annotations, on the other hand, overlay highlighted boxes, underlines, or arrows in specific visual areas of the presentation page to visually emphasize specific keywords being explained. Their display area aligns with the phoneme interval of the keyword; that is, the annotation appears when the keyword begins to be pronounced in the audio data and disappears when the pronunciation ends. By binding the display interval at the phoneme level, dynamic annotations can be precisely synchronized with the pace of the explanation.

[0076] Both types of generation tasks take the sequence of explanatory fragments as input. The data generation module can execute these two types of tasks in parallel to improve processing efficiency.

[0077] Because the visual area coordinates of dynamic annotation events are stored in page percentage coordinate format, when the output video resolution changes (for example, when the user requests that the micro-lesson video be adjusted from 1080p resolution to 720p resolution), the system does not need to perform coordinate transformation. It only needs to recalculate the pixel position of the annotation area according to the new screen pixel size and percentage coordinates to ensure that the annotation area accurately points to the same visual area in the courseware page and will not be offset or misaligned due to resolution changes.

[0078] For example, a product launch micro-course includes a bar chart comparing the sales of three product models. When explaining the point that "Model A has the highest sales," the system generates a dynamic annotation event based on the source page number (page 3) and visual region coordinates (the coordinates of the area corresponding to the "Model A" bar in the bar chart, recorded as percentage coordinates) in the semantic unit bound to the explanation segment. The display range of this annotation event coincides with the pronunciation range of the keyword "Model A." A highlighted box and a pointing arrow appear above the Model A bar in the bar chart when "Model A" begins to be pronounced, and the annotation automatically disappears when "Model A" ends to be pronounced. When the user downloads the video from 1080p resolution to 720p resolution, the system directly converts the resolution to the new pixel position using percentage coordinates, and the highlighted box still accurately points to the Model A bar area at the new resolution.

[0079] S4: Drive the digital human to generate a digital human picture track containing lip-sync sequences and body movements based on the phoneme time markers and prosodic markers.

[0080] The digital human driving module receives the phoneme time markers and prosodic markers generated in step S3, and uses them to drive the digital human to generate video tracks. For narration segments whose content remains unchanged, the corresponding phoneme time markers and prosodic markers remain unchanged, and the digital human video tracks can directly reuse the results generated initially. For narration segments whose speech data has been regenerated in step S3 (i.e., segments whose speech generation has been re-executed due to rewriting of the narration text), their phoneme time markers and prosodic markers have been updated, and the digital human driving module needs to regenerate the corresponding lip-sync sequences and body movements for these segments. The digital human video tracks for other segments remain unchanged. Specifically, the driving of the digital human is divided into the following two aspects:

[0081] Firstly, the generation of lip-shape sequences. Lip-shape sequences are generated based on phoneme time markers, and the digital human driving module maps the phoneme sequences to corresponding lip-shape units. Each phoneme corresponds to a lip shape; for example, the mouth opens wide when pronouncing the vowel / a: / , and the lips close when pronouncing the consonant / m / . The digital human driving module generates smooth transition animations between adjacent lip-shape units, making the lip-shape changes of the digital human appear natural and fluid. The time axis of the lip-shape sequence is directly aligned with the phoneme time markers, ensuring precise synchronization between the opening and closing changes of the digital human's lip shape and the speech playback. It should be noted that the technology of driving digital lip-shape animation based on phoneme sequences is a well-known technology in this field. For example, the Chinese patent "Digital Human Speech Synthesis and Lip-Shaping Synchronization Method and System" (CN122067510B) discloses a technical solution for generating continuous lip movements by extracting phoneme duration and fundamental frequency to form a prosodic state and determining the time correspondence between phonemes across segments; the academic paper "Model See Model Do: Speech-Driven FacialAnimation with Style Control" points out the correspondence between phonemes and visual pixels (lip movements) and continues to be applied in the field of speech-driven facial animation. Therefore, it will not be elaborated further here.

[0082] Secondly, the generation of body movements is based on prosodic markers combined with the semantic function of the explanatory sentences. The digital human-driven module determines the rhythm and timing of the movements based on the stress and pause information in the prosodic markers, and selects the type of movement according to the semantic function of the explanatory sentences. Specifically, defining sentences (such as "Safety production management refers to...") use indicative or expansive movements, and the digital human may make a gesture of palms facing upwards and outstretched; enumerating sentences (such as "It can be divided into the following three categories...") use counting movements, and the digital human may extend three fingers in sequence; warning sentences (such as "The following safety risks must be noted...") use emphatic movements, and the digital human may make a fist-clenching or pointing gesture. When the semantic unit bound to the explanatory segment contains visual region coordinates, the digital human-driven module will also generate gaze shift and pointing movements, so that the digital human's gaze is directed towards the corresponding area of ​​the courseware page. The technology of driving digital human body movements based on speech prosody and semantic functions is also a well-known technology in this field. For example, the Nebula platform launched by Mofa Technology has realized the direct generation of complete 3D digital humans containing speech, movements, expressions, and gestures from text. After semantic understanding of the input text through a large language model, it synchronously generates speech parameters, 3D skeletal movements, facial expressions, and hand gestures. Motion transfer models such as Audio2Motion and StyleGestures have also been widely used in the field of speech-driven body movement generation. Therefore, it will not be elaborated further here.

[0083] The digital human driving module generates a video track with a transparent background to facilitate subsequent overlay and compositing with the courseware footage. The video track also records composition areas and occlusion constraints to ensure the digital human does not obscure key content areas of the courseware pages.

[0084] S5: Based on the voice data, the sequence of explanatory segments, the courseware switching event, the subtitle event, the body movement segments in the digital human video track, and the dynamic annotation event, construct an anchor point event graph. Each anchor point in the anchor point event graph contains at least the event type, start and end time, and priority attributes. Furthermore, a sequential time constraint relationship is established between each anchor point according to the playback order of the explanatory segment sequence.

[0085] The timeline alignment module receives all media data generated in steps S3 and S4, including speech data and its phoneme and word boundaries, explanation segment sequences, courseware switching events, subtitle events, body movement segments in the digital human video track, and dynamic annotation events. Based on the phoneme and word boundaries in the speech data, the start and end boundaries of each explanation segment in the explanation segment sequence, the trigger time of the courseware switching event, the display range of the subtitle event, the start and end boundaries of the body movement segments in the digital human video track, and the display and disappearance times of the dynamic annotation events, it constructs an anchor point event graph.

[0086] Reference Figure 4As shown, the specific construction process of the anchor event graph is as follows:

[0087] S51, Anchor Point Extraction: Extract key time points from the media track data and create corresponding anchor points. The timeline alignment module iterates through the sequence of narration segments generated in step S2, creating a narration segment anchor point for each segment. The start and end times of the anchor point are taken from the planned playback interval of the narration segment on the overall timeline (determined by the target lesson allocation). For each phoneme boundary in the audio data, a phoneme anchor point is created, with the start and end times taken from the phoneme time markers generated in step S3. For each word boundary time marker in the audio data, a word boundary anchor point is created, with the start and end times taken from the word boundary time markers generated in step S3. For each courseware switching event, a courseware switching anchor point is created, with the anchor point's touch... The timing of the event is taken from the planned time of the courseware switching event; for each subtitle event, a subtitle anchor point is created, and the display range of the anchor point is taken from the time range of the word bound to the subtitle; for each limb action segment in the digital human video track, a digital human action anchor point is created, and the start and end times of the anchor point are taken from the planned playback range of the action segment (determined by the duration of the action); for each dynamic annotation event, a dynamic annotation anchor point is created to constrain the display and disappearance times of the dynamic annotation event, such as the display and disappearance times of the annotation graphic on the courseware page, and the display and disappearance times of the anchor point are taken from the phoneme range of the keyword bound to the annotation.

[0088] S52. Anchor Point Merging and Constraint Establishment: Incorporate all anchor points into the anchor point event graph and establish sequential timing constraints between each anchor point. After completing the above traversal, the timeline alignment module will uniformly incorporate all created anchor points into the anchor point event graph and establish sequential timing constraints between each anchor point according to the playback order of the narration segment sequence (i.e., determine the relative position of each anchor point on the timeline, and which anchor points must precede or follow which anchor points).

[0089] An anchor event graph is a data structure that uniformly models key time points on all media tracks as "anchor points." Each anchor point contains at least three attributes: event type (indicating the type of event the anchor point belongs to—phoneme, narration segment, word boundary, courseware switching, subtitle, action, or annotation), start and end time (the start and end positions of the event on the timeline), and priority (the importance of the anchor point in the alignment operation). The priority attribute value is preset by the system according to the anchor point type, with the physical acoustic boundary of the voice data as the highest reference benchmark. The higher the time accuracy requirement of the media track and the stronger its dependence on voice synchronization, the higher the priority of its corresponding anchor point. In a specific embodiment, the priorities of each type of anchor point are set from high to low as follows: phoneme anchor point > word boundary anchor point > narration segment anchor point > subtitle anchor point > dynamic annotation anchor point > digital human action anchor point > courseware switching anchor point. In this system, the phoneme anchor point serves as the primary anchor point, and its temporal position is fixed as the baseline for all subsequent alignment operations, with no deviation allowed. The offset tolerance range for other anchor points is asymmetrically allocated based on their priority level. For example, the offset tolerance for word boundary anchor points can be set to ±20 milliseconds, for narration segment anchor points to ±100 milliseconds, and for auxiliary anchor points such as courseware switching anchor points to ±300 milliseconds. The priority ranking is based on the temporal binding strength between each media track and the phoneme anchor point. The specific hierarchy and tolerance values ​​of the priority attributes can be configured by the system during initialization based on the total target lesson length and video frame rate for use in subsequent alignment operations. Furthermore, a sequential temporal constraint relationship is established between each anchor point according to the playback order of the narration segment sequence. For example, narration segment anchor point A must be after courseware switching anchor point B and before subtitle anchor point C. This sequential constraint is explicitly recorded in the anchor point event graph.

[0090] Taking a micro-lesson containing three explanatory segments as an example, the anchor point event diagram includes: three explanatory segment anchor points corresponding to the three explanatory segments, each anchor point recording the start and end times of the segment; several phoneme anchor points corresponding to speech phonemes, each anchor point recording the start and end times of the phoneme; three courseware switching anchor points corresponding to three courseware page turns (courseware switching); subtitle anchor points corresponding to several subtitles; action anchor points corresponding to several digital human actions; and annotation anchor points corresponding to several dynamic annotations. All these anchor points are arranged in the playback order and establish a temporal constraint chain like "courseware switching anchor point — explanatory segment anchor point 1 — subtitle anchor point 1-1 — digital human action anchor point 1-1 — phoneme anchor point...". Among them, "subtitle anchor point 1-1" indicates that the subtitle anchor point belongs to the first explanatory segment and is the anchor point corresponding to the first subtitle in the explanatory segment; "digital human action anchor point 1-1" indicates that the digital human action anchor point belongs to the first explanatory segment and is the anchor point corresponding to the first action in the explanatory segment.

[0091] The anchor point event graph includes phoneme anchors, word boundary anchors, narration segment anchors, courseware switching anchors, subtitle anchors, digital human action anchors, and dynamic annotation anchors. The system extracts phoneme boundaries from the speech data and creates a phoneme anchor for each phoneme boundary. The start and end times of the phoneme anchor are the start and end times of that phoneme boundary. Phoneme anchors are used to constrain the timeline of the lip-sync sequence in the digital human's video track; that is, the lip-sync changes of the digital human must be precisely synchronized with the pronunciation time of the phoneme, and the lip shape must present the corresponding shape within the time interval of a given phoneme. The system extracts word boundary time markers from the speech data and creates a word boundary anchor for each word boundary time marker. The start and end times of the word boundary anchor are the start and end times of that word boundary time marker. Word boundary anchors are used to provide calibration references for subtitle display intervals and action start and end boundaries at the word level. The system extracts the start and end boundaries of each narration segment from the narration segment sequence and creates a narration segment anchor for each narration segment. The start and end times of the narration segment anchor are the planned playback interval of that narration segment on the overall timeline. The explanatory segment anchor points constrain the start and end times of each explanatory segment, ensuring that the segments are seamlessly connected, non-overlapping, and complete on the timeline. The system extracts the trigger time of each courseware switching event from the courseware switching events and creates a courseware switching anchor point for each event. The trigger time of the courseware switching anchor point is the planned time for that event. The courseware switching anchor point determines the target page for the courseware switching based on the source page number in the semantic unit bound to the explanatory segment and constrains the trigger time for switching to that target page. When an explanatory segment begins explaining the content of a page, the courseware switching anchor point instructs the courseware screen to switch to the corresponding page number at the start time of that segment. The system extracts the display range of each subtitle event from the subtitle events and creates a subtitle anchor point for each event. The display range of the subtitle anchor point is the time range of the word boundary bound to that subtitle. The subtitle anchor point constrains the display range of each subtitle on the timeline, ensuring that the subtitle display is synchronized with the corresponding audio playback. The system extracts the start and end boundaries of each limb movement segment from the digital human video track, creating a digital human movement anchor point for each segment. The start and end times of the digital human movement anchor point represent the planned playback interval of that movement segment (determined by the duration of the movement). The digital human movement anchor points constrain the start and end boundaries of each limb movement, ensuring that the start and end of the movement match the rhythm of the narration. The system also extracts the visibility and invisibility times of each annotation from dynamic annotation events, creating a dynamic annotation anchor point for each event. The visibility and invisibility times of the dynamic annotation anchor point represent the phoneme interval of the keyword bound to that annotation, and the time position of the dynamic annotation anchor point is directly constrained by the phoneme anchor point.

[0092] In practical applications, anchor event graphs contain various types of anchors, each playing a different role in the alignment operation. In this embodiment, phoneme anchors are used as primary anchors, while word boundary anchors and explanatory segment anchors are used as secondary anchors. Phoneme anchors are designated as primary anchors because their start and end times are directly derived from phoneme boundaries in the speech data. These phoneme boundaries are precisely generated acoustically by the speech synthesis engine, possessing the highest temporal resolution and reliability. Using them as primary anchors ensures that the entire timeline alignment operation is based on the most stable and accurate time reference. The primary anchor serves as the main time reference for local time alignment, while the secondary anchors further refine the start and end boundaries of media segments within the reference interval determined by the primary anchor.

[0093] By classifying anchor point types and assigning roles as described above, the anchor point event graph not only records the time position of each media event, but also clarifies the constraint relationships between various anchor points, providing a precise reference for subsequent local time alignment operations.

[0094] It should be noted that in this step, the sequential timing constraint and the priority attribute are two different dimensions of constraints in the anchor event graph: the sequential timing constraint defines the order of each anchor point on the playback timeline (for example, the courseware switching anchor point must precede the explanation segment anchor point), ensuring the correctness of the playback logic; the priority attribute defines the stability of the time position of each anchor point in the alignment operation (for example, the phoneme anchor point has the highest priority and is not allowed to shift), ensuring the precision control of the alignment operation.

[0095] For example, a company's training department uses this system to create a micro-lesson video on "Safety Production Management System". In step S5, the system constructs an anchor event graph for this micro-lesson: approximately 200 phoneme anchors, precisely corresponding to each phoneme output by the speech synthesis engine; approximately 80 word boundary anchors, corresponding to word boundary time markers in the speech data; 12 explanation segment anchors, corresponding to 12 explanation segments; 8 courseware switching anchors, corresponding to 8 courseware page turns (courseware switching); 36 subtitle anchors, corresponding to 36 subtitles; 15 digital human action anchors, corresponding to 15 digital human body actions; and 8 dynamic annotation anchors, corresponding to 8 key content annotations. In the subsequent S6 alignment step, the timeline alignment module uses 200 phoneme anchor points as the highest priority benchmark. Based on this, it sequentially calibrates the boundaries of 12 narration segment anchor points, the trigger times of 8 courseware switching anchor points, the display range of 36 subtitle anchor points, the start and end boundaries of 15 digital human action anchor points, and the display and disappearance times of 8 annotation anchor points. Strict temporal constraints are established between each anchor point according to the playback order to ensure precise synchronization between lip movements, speech, page turns, subtitles, actions, and annotations throughout the video playback.

[0096] In the practical application of anchor event graphs, the applicant discovered that the time offset of one anchor point could propagate along dependencies to other anchor points. To precisely control this propagation effect and minimize the scope of realignment,

[0097] In this embodiment of the invention, the anchor event graph also records the dependency propagation paths between anchors. Each dependency propagation path represents the subsequent anchors affected by a shift in the start and end times of an anchor, originating from that anchor. A dependency propagation path is a directed relationship that defines which subsequent anchors are affected when the time position of an anchor changes. After the anchor event graph is constructed, the dependency propagation paths are automatically generated by the timeline alignment module based on the sequential temporal constraints between anchors and the dependencies of the media content. Specifically, when a shift in the time position of anchor A directly affects the time position of anchor B, a directed dependency propagation edge from A to B is added to the anchor event graph. For example, a time shift in the narration segment anchor will cause a time shift in all its internal phoneme anchors, subtitle anchors, and dynamic annotation anchors. Therefore, a directed dependency propagation edge from the narration segment anchor to each of its internal phoneme anchors, subtitle anchors, and dynamic annotation anchors is recorded in the anchor event graph. The dependency propagation path is generated once when the anchor event graph is initially built. When the anchor event graph changes (such as when anchors are added or deleted in an incremental update), the dependency propagation path of the affected anchors is recalculated only.

[0098] For example, in a typical dependency relationship: courseware switching anchor point — explanation segment anchor point — phoneme anchor point — lip-sync anchor point. This means that if the timing of the courseware switching anchor point shifts (e.g., due to page content adjustments causing page turn times to change), the dependency propagation path will indicate that the shift will propagate along the path to the explanation segment anchor point (the start time of the explanation segment needs to be adjusted accordingly), the phoneme anchor point (the start time of the speech needs to be adjusted accordingly), and the lip-sync anchor point (the start time of the lip movements needs to be adjusted accordingly). However, the subtitle anchor point, if not on this dependency propagation path, remains unaffected and unchanged.

[0099] When performing local time alignment, if a time shift occurs at a certain anchor point, the timeline alignment module first queries the dependency propagation path recorded in the anchor point event graph to determine all subsequent anchor points affected by the shift. Then, it only performs realignment operations on these affected subsequent anchor points, while anchor points outside the dependency propagation path retain their original time positions. This precise impact range definition mechanism avoids triggering a global realignment due to the adjustment of a single anchor point. It should be noted that "calibration" in step S5 refers to fine-tuning the anchor point time within the same anchor point event graph, while "realignment" in step S6 refers to re-performing the alignment operation after a change in the anchor point event graph (due to material updates or anchor point shifts). The two differ in level; calibration is a fine-tuning within a single alignment operation, while realignment is a realignment during an iterative process.

[0100] For example, after a compliance system micro-course video is generated, users may find that the start time of the explanation segment on page 5 needs to be delayed by 0.5 seconds. The timeline alignment module locates the corresponding explanation segment anchor point in the anchor event graph and shifts its start and end times forward by 0.5 seconds. Then, the module queries along the dependency propagation path: the subsequent anchor points of this explanation segment anchor point include the phoneme anchor point, lip-sync anchor point, and dynamic annotation anchor point corresponding to page 5. The module only performs realignment on these subsequent anchor points, shifting the phoneme anchor point and its corresponding audio segment forward by 0.5 seconds, the lip-sync anchor point and its corresponding lip-sync animation forward by 0.5 seconds, and the dynamic annotation anchor point and its corresponding annotation visibility time forward by 0.5 seconds. The courseware switching anchor points before page 5, other explanation segment anchor points after page 5, and subtitle anchor points on other pages are not on the dependency propagation path and remain unchanged. The entire realignment operation only involves 3 anchor points and their corresponding media segments, avoiding the need to regenerate the entire video.

[0101] S6: Based on the priority and sequential timing constraints of each anchor point recorded in the anchor point event diagram, and using the phoneme boundaries in the speech data as the time reference, perform local time alignment on the corresponding courseware switching events, subtitle events, action segments in the digital human video track, and dynamic annotation events, and output the synthesized timeline.

[0102] The timeline alignment module uses the anchor event graph constructed in step S5 as a basis to perform multi-track timeline alignment operations. The alignment operation first follows sequential timing constraints to ensure correct playback order. Then, it controls the offset of each anchor point based on priority attributes to achieve optimal synchronization while maintaining logical order. Specifically, the phoneme boundaries in the audio data are used as the most reliable time reference (i.e., the main anchor point). After the phoneme boundaries are determined, the timeline alignment module adjusts the event time positions of each media track sequentially according to the priority of each anchor point from high to low. The "local time alignment" in this step refers to dividing the entire timeline into several independent time intervals using the start and end times of each anchor point in the anchor event graph as boundaries, and performing time calibration operations independently within each interval. The boundaries of these independent intervals are defined by anchor points with stable time positions (such as phoneme anchor points). These boundary anchor points do not shift during version iterations or local adjustments, thus ensuring that the playback timing of adjacent intervals does not interfere with each other.

[0103] Specifically, the start and end times of the highest priority anchors (such as phoneme anchors) are fixed as the baseline and are not allowed to shift. The next highest priority anchors (such as narration segment anchors) can have their start and end times fine-tuned within an allowed offset range. Lower priority anchors (such as dynamic annotation anchors) are adjusted within the time interval defined by the next highest priority anchor. Constraints on the alignment operation include: the sequential relationship recorded in the anchor event graph must be satisfied. For example, a courseware switching anchor must occur before the start of its corresponding narration segment anchor, a subtitle anchor must be within the time interval of its corresponding narration segment anchor, and the appearance and disappearance times of dynamic annotation anchors must be consistent with the phoneme interval of the corresponding keyword.

[0104] Using the aforementioned local time alignment method, events on each media track are precisely adjusted to the time interval specified by their corresponding anchor points, outputting a unified multi-track composite timeline. This composite timeline records the synchronized playback moments of each media track in each frame.

[0105] In local time alignment operations, anchors of different priorities should have different offset tolerances to ensure the stability of high-precision anchors (such as phoneme anchors).

[0106] Therefore, during local time alignment, the time position of each event or segment is adjusted to within the time interval specified by its corresponding anchor point, with higher-priority anchor points having smaller offsets than lower-priority anchor points. Each anchor point records its specified time interval in the anchor point event graph, which is the ideal time range that the event or segment corresponding to that anchor point should occupy. The goal of the alignment operation is to adjust the actual time position of each event or segment to within this specified interval.

[0107] Priority attributes determine the tolerance range of offsets during alignment operations. High-priority anchors (such as phoneme anchors) have smaller allowable offsets; for example, the allowable offset for a phoneme anchor might be set to ±10 milliseconds, meaning the start and end times of a phoneme anchor can only be adjusted by a maximum of 10 milliseconds. Low-priority anchors (such as dynamic annotation anchors) have larger allowable offsets; for example, the allowable offset for a dynamic annotation anchor might be set to ±200 milliseconds. This rule of "high-priority anchor offsets being smaller than low-priority anchor offsets" ensures that timeline alignment operations provide sufficient adjustment space for low-precision anchors while maintaining the stability of high-precision anchors.

[0108] Through the aforementioned priority and offset constraint mechanism, the timeline alignment module can flexibly adjust the timing of auxiliary tracks such as courseware page turning (courseware switching), subtitles, and annotations while ensuring the precise synchronization of key media tracks such as voice and lip movements, thereby optimizing the overall synchronization effect.

[0109] S7: Output the micro-lesson video according to the synthesized timeline.

[0110] The video output module renders and composites the digital human video track, courseware video track, subtitle track, and dynamic annotation track based on the composite timeline output in step S6. The composite timeline is a unified multi-track time mapping table that records the precise position of each event on the timeline for each media track—the playback time of each frame in the digital human video track, the switching and pause times of each page in the courseware video track, the display interval of each subtitle in the subtitle track, and the appearance and disappearance times of each annotation in the dynamic annotation track. The courseware switching events, subtitle events, and dynamic annotation events generated in step S3, after being time-calibrated according to the composite timeline in step S6, drive the corresponding courseware video switching, subtitle text display, and annotation graphic appearance and disappearance, forming the courseware video track, subtitle track, and dynamic annotation track that participate in the final rendering. These tracks, along with the digital human video track generated by the digital human driving module, participate in the final rendering and compositing. Specifically, the video output module overlays and synthesizes the digital human video track (with a transparent background) and the courseware video track. The digital human is placed as the foreground in a preset position on the courseware screen (such as the lower right or lower left corner) to ensure that it does not obscure the key content areas of the courseware. The subtitle track is displayed sequentially at the bottom of the screen according to the timeline. The dynamic annotation track displays highlighted boxes, underlines, or arrows in the corresponding areas of the courseware page at specified times. It should be noted that the above process of overlaying and synthesizing multiple video tracks into the final video is a well-known technology in the field (see Chinese patent CN114741541A "Method and Device for Interactive Control of PPT by AI Digital Human Based on Template Editing", which discloses a technical solution for adding AI digital human video frames and page images to the playback template for display, rendering to generate image frames, and outputting them synchronously with audio frames; see also CN119603415A "A Video Synthesis Method, Device, Electronic Device and Storage Medium", which discloses a technical solution for video synthesis of the courseware video to be synthesized and the digital human model based on digital human layout data). The video output module ultimately generates the micro-lesson video file, along with accompanying subtitle files and a timeline list for users to review and edit later.

[0111] Through steps S1 to S7 described above, this invention automatically transforms an original PPT, PDF, or Word document into a micro-lesson video narrated by a digital human, including synchronized subtitles and dynamic annotations. Throughout the process, the semantic unit graph provides traceable material constraints for the generation of the narration, while the anchor event graph provides a unified spatiotemporal modeling framework for multi-track alignment. Together, these two technologies solve the problems of poor content traceability and low multi-track synchronization accuracy in existing technologies.

[0112] In practical use, the content of the input courseware often changes, such as policy updates, product parameter adjustments, and operational process optimizations. To update the generated micro-lesson videos with minimal cost after these changes occur, this embodiment provides the following preferred implementation method in addition to the main processes S1 to S7. This implementation method is triggered and executed after a version change occurs in the input courseware.

[0113] When the micro-lesson video is initially generated (i.e., during steps S1 to S7), the system establishes a unique content fingerprint for each traceable granular unit. The content fingerprint is a fixed-length hash value generated by hashing the content data of the granular unit using a cryptographic hash algorithm (such as SHA-256 or MD5). Whenever the content of the unit changes, its fingerprint changes. Specifically, the system establishes a unique content fingerprint for each page element in the page element tree, each semantic unit in the semantic unit graph, each explanation segment in the explanation segment sequence, each audio segment in the audio data, each digital human segment in the digital human video track, and each media synthesis segment in the synthesis timeline. Simultaneously, the system records the upstream dependencies between each content fingerprint. These upstream dependencies are recorded in the form of a dependency table, which contains at least three fields: upstream node identifier, downstream node identifier, and dependency type. The dependency type includes types such as "semantic unit—explanation segment," "explanation segment—audio segment," "explanation segment—digital human segment," and "audio segment / digital human segment—media synthesis segment." The dependency table can be stored in a separate data table in the system database. When any node is marked as failed, the system propagates the failure mark downstream by querying this table.

[0114] When the input courseware changes version, for example, when a user uploads an updated PPT file, an incremental update operation is triggered. The system first compares the content fingerprints of each page element before and after the change, identifying the page elements whose fingerprints have changed. Then, the system marks the semantic units corresponding to these page elements as invalid semantic units. Next, the system propagates the invalidation markers downstream along the upstream dependencies: invalid semantic units cause the narration segments that depend on them to become invalid, invalid narration segments cause the audio segments and digital human segments that depend on them to become invalid, and invalid audio segments and digital human segments cause the media synthesis segments that depend on them to become invalid. Through this hierarchical propagation, the system accurately identifies all affected nodes.

[0115] Finally, the system only regenerates the explanatory segments, audio segments, digital human segments, and media synthesis segments marked as failed nodes. The regenerated segments undergo local time alignment between adjacent unfailed anchor points. These unfailed anchor points serve as stable time boundaries, ensuring that the regenerated segments can be seamlessly embedded into the original timeline without affecting the playback sequence of the unchanged segments.

[0116] Through the incremental update mechanism described above, when the input courseware undergoes partial changes, the system only regenerates the smallest set of affected segments, avoiding the high cost of regenerating the entire video.

[0117] For example, a company's compliance policy PowerPoint presentation has 20 pages, and the system has already generated a 10-minute micro-lecture video for it. Three months later, on page 8 of the policy, an approval amount was changed from "500,000 yuan" to "1,000,000 yuan". The user uploaded the updated PowerPoint file. The system compared the content fingerprints of elements on each page before and after the change and found that only the fingerprint of the text element corresponding to this amount on page 8 had changed. The system marked the semantic unit corresponding to this text element as a invalid semantic unit and then propagated it along the dependency relationship: the explanation segment corresponding to this semantic unit became invalid, the audio segment and the digital human segment corresponding to this explanation segment became invalid, and the media synthesis segment corresponding to this audio segment and the digital human segment became invalid. The system only regenerated this explanation segment, audio segment, digital human segment, and media synthesis segment, and performed local time alignment between the non-invalid anchors before and after the explanation segment. The content of the remaining 19 pages and the more than 9 minutes of video clips remained unchanged. The entire update process was completed in seconds, while regenerating the complete video using traditional methods would have taken much longer.

[0118] During incremental updates, some changes may involve more than just numerical or textual modifications; they may involve adjustments to page layout or visual structure. Therefore, after marking explanatory segments, audio segments, digital human segments, and media synthesis segments directly or indirectly associated with the failed semantic unit as failed nodes along the upstream dependencies, when the page containing the source page number of a marked failed semantic unit is also marked as failed, page re-parsing of that page is triggered. After page re-parsing, if the change in the visual region coordinates of the marked failed semantic unit before and after the version change exceeds a preset threshold, all explanatory segments and downstream media segments of each semantic unit dependent on the page corresponding to that source page number are regenerated. The change can be measured using Euclidean distance (calculating the offset distance between coordinate positions) or intersection-union ratio (calculating the degree of overlap between two rectangular regions before and after the change).

[0119] When using Euclidean distance as a metric, if the offset distance between coordinate positions exceeds a preset threshold (e.g., Euclidean distance exceeding 100 pixels), or when using intersection-union ratio (IU) as a metric, if the overlap between the two rectangular regions before and after the change is less than a preset threshold (e.g., IU is less than 0.5), it indicates that the position of the knowledge point on the page has changed significantly. This could be due to page layout adjustments, chart relocation, or image replacement. In this case, simply regenerating the explanation segments, audio segments, and digital human segments is insufficient, because the structure of the courseware page itself has changed, and the original semantic unit diagram may no longer be accurate.

[0120] Therefore, the system triggers a page re-parsing of the page containing the source page number corresponding to the semantic unit. That is, the system re-executes the parsing operation in step S1 on that page: re-extracting the page elements and visual regions, and reconstructing the semantic unit graph corresponding to the page. After the page re-parsing is complete, the system regenerates all explanatory segments and downstream media segments (audio segments, digital human segments, and media synthesis segments) that depend on the semantic unit corresponding to that page, and performs local time alignment between adjacent, undamaged anchor points.

[0121] Through the aforementioned cross-media verification mechanism, the system can not only handle changes in text content, but also respond to adjustments in page layout and visual structure, ensuring the integrity and accuracy of incremental updates.

[0122] For example, page 5 of a product launch presentation originally contained a bar chart showing sales data for three products. In a version update, this bar chart was replaced with a line chart showing the trend of the same data. Although the data itself did not change much, the visual region coordinates changed significantly; the coordinate positions of the bars in the bar chart were completely different from the coordinate positions of the data points in the line chart. During the incremental update, the system detected that the intersection-union ratio (IUU) of the visual region coordinates of the semantic unit corresponding to page 5 was lower than a preset threshold (e.g., 0.3), indicating that the overlap between the two rectangular regions before and after the change was significantly insufficient, suggesting a substantial change in the page layout. The system triggered a page re-parsing of page 5, re-identifying the visual regions and teachable elements of the page, and reconstructing the semantic unit graph corresponding to page 5. Then, the system regenerated all the explanation segments (2 in total), audio segments (2), digital human segments (2), and media synthesis segments (1) that depended on the semantic unit of page 5, and completed local time alignment between adjacent non-failed anchor points. In the updated video, the explanation on page 5 matches the new line chart precisely, and the dynamic annotations accurately point to the corresponding data points in the line chart.

[0123] Example 3

[0124] Combination Figure 5As shown in the illustration, this disclosure also provides an electronic device for an automatic micro-lesson video generation method based on anchor event graphs, including a processor 300 and a memory 301. Optionally, the system may further include a communication interface 302 and a bus 303. The processor 300, communication interface 302, and memory 301 can communicate with each other via the bus 303. The communication interface 302 can be used for information transmission. The processor 300 can call logical instructions in the memory 301 to execute the automatic micro-lesson video generation method based on anchor event graphs described above.

[0125] Furthermore, the logic instructions in the aforementioned memory 301 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0126] The memory 301, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 300 executes functional applications and data processing by running the program instructions / modules stored in the memory 301, thereby implementing the micro-lesson video automatic generation method based on anchor event graphs in the above embodiments.

[0127] The memory 301 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 301 may include high-speed random access memory and may also include non-volatile memory.

[0128] The foregoing description and accompanying drawings fully illustrate embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in the embodiments of this disclosure is for descriptive purposes only and is not intended to limit the claims. As used in the description of the embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in the embodiments of this disclosure refers to any and all possible combinations of one or more of the associated listed elements. Additionally, when used in embodiments of this disclosure, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, then the relevant parts can be referred to the description of the method section.

[0129] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

Claims

1. A method for automatically generating micro-lesson videos based on anchor point event graphs, characterized in that, The method includes: Obtain input courseware, construct a semantic unit graph based on the input courseware, wherein the semantic units in the semantic unit graph have a logical order, and each semantic unit is associated with a teaching knowledge point in the input courseware; A sequence of explanatory segments is generated based on the logical order between semantic units, and at least one semantic unit identifier is bound to each explanatory segment in the sequence of explanatory segments. Based on the sequence of explanatory segments, speech data and corresponding phoneme time markers, word boundary time markers, and prosodic markers are generated, as well as courseware switching events, subtitle events, and dynamic annotation events. The digital human is driven to generate a digital human video track containing lip-sync sequences and body movements based on the phoneme time markers and the prosodic markers. Construct an anchor event graph, wherein each anchor point in the anchor event graph contains at least an event type, start and end time, and priority, and establish a sequential time constraint relationship between each anchor point according to the playback order of the explanation segment sequence; Based on the sequential temporal constraints of the anchor event graph and the priority of each anchor point, the events corresponding to each anchor point are locally time-aligned using the phoneme boundaries in the speech data as the time reference, and a synthesized time axis is output. The micro-lesson video is output based on the synthesized timeline.

2. The method according to claim 1, characterized in that, The construction of the semantic unit graph based on the input courseware includes: parsing the input courseware into a page element tree and a visual region tree; constructing the semantic unit graph based on the page elements in the page element tree and the visual regions in the visual region tree; each semantic unit includes at least the source page number, original text range, concept tag, and visual region coordinates of the knowledge point in the input courseware; the anchor point event graph includes phoneme anchors, word boundary anchors, explanation segment anchors, courseware switching anchors, subtitle anchors, digital human action anchors, and dynamic annotation anchors; when constructing the anchor point event graph, the phoneme anchors are used as the main anchors, and the word boundary anchors and explanation segment anchors are used as auxiliary anchors.

3. The method according to claim 2, characterized in that, The anchor event graph also records the dependency propagation path between each anchor point. The dependency propagation path is used to characterize one or more subsequent anchor points affected by the time offset of an anchor point. When local time alignment is performed and an anchor point experiences a time offset, the affected subsequent anchor points are determined along the dependency propagation path, and realignment is performed only on the affected subsequent anchor points.

4. The method according to claim 2, characterized in that, The dynamic annotation event is generated based on the source page number, visual region coordinates, and concept tags in the semantic unit bound to the explanation segment. The source page number is used to determine the target page where the annotation is located, the visual region coordinates are used to determine the display position of the annotation on the target page, and the concept tags are used to determine the keyword to be annotated. A word matching the keyword is searched in the word boundary time stamp, and the phoneme time interval corresponding to the word in the speech data is used as the display interval of the dynamic annotation event, so that the display interval of the dynamic annotation event is consistent with the phoneme interval of the keyword.

5. The method according to claim 2, characterized in that, When performing local time alignment, the time position of each event or segment is adjusted to the time interval specified by its corresponding anchor point, and the offset of the anchor point with higher priority is less than the offset of the anchor point with lower priority.

6. The method according to claim 2, characterized in that, After generating the sequence of explanatory segments, the method further includes: Calculate the key concept coverage rate of the semantic units referenced by each explanatory segment relative to all effective semantic units in the semantic unit graph, and the source support rate of the number of sentences in each explanatory segment whose content is supported by the original text range of the semantic units to which they are bound relative to the total number of sentences in that explanatory segment; When the coverage of the key concepts is lower than the first threshold, the semantic units that are not referenced by any explanatory segments are marked as uncovered semantic units, and the explanatory segments corresponding to the uncovered semantic units are regenerated. When the source support rate is lower than the second threshold, sentences in the explanation segment that lack original text range support are marked as sourceless sentences, and explanation segments containing sourceless sentences are regenerated. After regeneration, coverage and support are recalculated until both reach their respective thresholds.

7. The method according to claim 2, characterized in that, The method also performs an incremental update operation, which is triggered after the input courseware undergoes a version change, including: When the micro-lesson video is generated for the first time, a content fingerprint and upstream dependency relationship are established, and the relevant content fingerprint is updated according to the changes after the version is changed. The content fingerprint includes at least the unique fingerprint corresponding to each page element in the page element tree, each semantic unit in the semantic unit graph, each explanation segment in the explanation segment sequence, each speech segment in the speech data, each digital human segment in the digital human video track, and each media synthesis segment in the synthesis timeline. When the input courseware is changed, the content fingerprints of each page element before and after the change are compared. The semantic units corresponding to the page elements whose content fingerprints have changed are marked as invalid semantic units. The explanation segments, audio segments, digital human segments and media synthesis segments that are directly or indirectly related to the invalid semantic units are marked as invalid nodes along the upstream dependency relationship. Regeneration is performed only on explanatory segments, speech segments, digital human segments, and media synthesis segments marked as invalid nodes, and local time alignment is performed on the regenerated segments between adjacent non-invalid anchor points.

8. The method according to claim 7, characterized in that, After marking the explanatory segments, speech segments, digital human segments, and media synthesis segments that are directly or indirectly associated with the failed semantic unit as failed nodes along the upstream dependency relationship, the method further includes: When the page containing the source page number of a semantic unit marked as invalid is also marked as invalid, page re-parsing of that page is triggered. After page re-parsing, if the change in the visual region coordinates of the semantic unit marked as invalid exceeds a preset threshold before and after the version change, then all explanatory segments and downstream media segments of each semantic unit that depend on the page corresponding to the source page number will be regenerated.

9. A micro-lesson video automatic generation system based on anchor point event graphs, characterized in that, include: The semantic unit graph construction module acquires input courseware and constructs a semantic unit graph based on the input courseware. The semantic units in the semantic unit graph have a logical order, and each semantic unit is associated with a teaching knowledge point in the input courseware. The explanation segment sequence generation module generates an explanation segment sequence according to the logical order between semantic units, and binds at least one semantic unit identifier to each explanation segment in the explanation segment sequence. The data generation module is used to generate speech data and corresponding phoneme time markers, word boundary time markers and prosodic markers according to the sequence of the explanation segments, as well as to generate courseware switching events, subtitle events and dynamic annotation events; The digital human driving module is used to drive the digital human to generate a digital human picture track containing lip-sync sequences and body movements based on the phoneme time markers and prosodic markers. The timeline alignment module is used to construct an anchor event graph based on the voice data, the sequence of explanatory segments, the courseware switching events, the subtitle events, the body movement segments in the digital human video track, and the dynamic annotation events. Each anchor point in the anchor event graph contains at least the event type, start and end time, and priority. Furthermore, a sequential time constraint relationship is established between each anchor point according to the playback order of the explanatory segment sequence. And based on the sequential timing constraints of the anchor point event graph and the priority of each anchor point, using the phoneme boundaries in the speech data as the time reference, perform local time alignment on the corresponding courseware switching events, subtitle events, body movement segments in the digital human video track, and dynamic annotation events, and output a synthesized timeline; The video output module is used to output micro-lesson videos according to the synthesized timeline.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the processor to perform the method of any one of claims 1 to 8.

Citation Information

Patent Citations

  • Method and device for interaction control of AI digital human to PPT based on templated editing

    CN114741541A

  • Video synthesis method and device, electronic equipment and storage medium

    CN119603415A

  • Digital human voice synthesis and lip synchronization method and system

    CN122067510B