A multimodal large model driven video-to-presentation automatic generation method

Abstract

The present application relates to the technical field of electric digital data processing, in particular to a multi-modal large model driven video-to-PPT automatic generation method, which first acquires a target video network link, completes download analysis and speech recognition transcription, corrects the segmentation boundary by using voice activity detection, screens invalid voice segments and divides semantic breakpoints; then inputs the valid voice segments into a multi-modal large model, completes theme analysis, semantic coding and correlation comparison relying on threshold verification, clusters to generate semantic units and page boundaries; performs two-dimensional compliance verification of the semantic units in terms of duration and text, automatically merges or splits the units, perfects the page core points, and can also adaptively adjust the page judgment rules according to the scene. The present application solves the problems of mechanical segmentation, no semantic understanding and low automation degree of the traditional scheme, realizes intelligent generation of the whole process, and is suitable for multiple application scenarios of explanation videos.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, and more specifically, to a method for automatically generating video-to-presentation conversion driven by a multimodal large model. Background Technology

[0002] With the explosive growth of online knowledge popularization, vocational training, and online education video resources, the ability to quickly convert explanatory video content into structured presentations has become a necessity for knowledge dissemination and lesson preparation. Traditional production methods rely on manual viewing of videos, text organization, image extraction, and manual PPT page division, which is time-consuming, labor-intensive, and the quality of page division is greatly affected by human subjective experience. Existing conventional video-to-PPT solutions mostly use fixed-duration, fixed-word mechanical text segmentation, lacking a deep semantic understanding of the video's theme and knowledge level, resulting in rigid page boundaries that are disconnected from the actual explanation logic. At the same time, ordinary auxiliary tools only have simple speech-to-text functions and cannot perform validity screening, timing correction, or semantic breakpoint division of speech segments. They also cannot achieve multimodal information fusion and threshold compliance verification, making it difficult to adapt to page division rules adjustments for different application scenarios. They generally suffer from poor image-text matching, low automation, and the need for a lot of manual intervention, failing to meet the application needs of generating presentations efficiently and in a standardized manner from large batches of videos. Summary of the Invention

[0003] In view of this, in order to address the shortcomings of the prior art, the present invention proposes a method for automatically generating video-to-presentation driven by a multimodal large model, aiming to solve at least one of the problems mentioned in the background art.

[0004] This invention provides a method for automatically generating video-to-presentation conversion driven by a multimodal large model, comprising the following steps: obtaining the target video network link and completing the video file download and format parsing, performing machine speech recognition and transcription on the entire video audio, and outputting coarse segmentation data of the original audio text carrying timestamp information; The speech activity detection model is used to correct the start and end time boundaries of the coarse segmentation of the original speech text. Invalid speech segments are filtered out by combining the text character length. Then, temporary semantic breakpoints are divided according to the time interval between adjacent speech segments and continuous semantic segments are merged. The selected valid speech and text segments are input into a multimodal large model to analyze the global theme and knowledge level of the video. The semantics of each speech segment are semantically encoded and the semantic association of adjacent segments is compared based on the theme validity. According to the preset semantic matching logic, highly related segments are clustered into the same semantic unit, and the connection points of semantically different segments are defined as the page boundaries of the presentation. Each semantic unit undergoes compliance verification based on both video duration and text length. Excessively short semantic units are merged into adjacent units with the same theme, while excessively long semantic units are further semantically split by a multimodal large model according to sub-knowledge points. Based on compliant semantic units, the presentation automatically generates page titles, key points, and standardized structured pagination data for each page.

[0005] In some embodiments, the step of obtaining the target video network link and completing video file download and format parsing, performing machine speech recognition and transcription on the entire video audio, and outputting raw audio-text coarse segmentation data carrying timestamp information includes: The system identifies video container format types in a preset priority order and determines whether the current video format is a system preset standard playback format. When the current video format is not the system's default standard playback format, the system will automatically perform format transcoding and adaptation processing to convert it to the standard video format before proceeding with the subsequent speech recognition and transcription operation. When the current video format is the system's preset standard playback format, the speech recognition and transcription process will proceed directly.

[0006] In some embodiments, the step of using a speech activity detection model to correct the start and end time boundaries of the coarse segmentation of the original speech text, filtering out invalid speech segments based on the text character length, and then dividing temporary semantic breakpoints and merging continuous semantic segments according to the time interval between adjacent speech segments includes: Determine whether the number of characters in a single audio text segment meets the system's preset character standard and whether the proportion of redundant text tone exceeds the preset ratio; When the number of characters in a single audio-text segment does not reach the preset character standard and the proportion of redundant tone components exceeds the preset ratio, the segment is determined to be a meaningless tone fragment and is directly filtered out. When the number of characters in a single audio-text segment reaches the preset character standard and the proportion of redundant tone components does not exceed the preset ratio, it is determined to be a valid audio segment and retained for subsequent processing.

[0007] In some embodiments, when the step of using a speech activity detection model to correct the start and end time boundaries of the coarse segmentation of the original speech text, filtering out invalid speech segments based on the text character length, and then dividing temporary semantic breakpoints and merging continuous semantic segments according to the time interval between adjacent speech segments, the step further includes: Determine whether the time interval between adjacent valid speech segments exceeds the system's preset interval range, and simultaneously determine whether the speech pause attribute is a short pause for natural explanation or a long pause for knowledge point switching; When the time interval between adjacent valid speech segments exceeds the preset range and is determined to be a long pause in knowledge point switching, then that point is marked as a formal semantic breakpoint. When the time interval between adjacent valid audio segments is within a preset range and is determined to be a natural short pause in the explanation, it is determined to be continuous explanation content, and the adjacent audio segments are temporarily merged into a whole segment.

[0008] In some embodiments, the step of inputting the filtered valid speech-text segments into a multimodal large model, parsing the global theme and knowledge level of the video, semantically encoding each speech segment based on theme validity, comparing the semantic associations of adjacent segments, clustering highly correlated segments into the same semantic unit according to a preset semantic matching logic, and defining the connection points of semantically different segments as the page boundaries of the presentation includes: Pre-set a minimum threshold for the number of knowledge levels and a threshold for topic completeness. Count the number of knowledge levels in the initial parsing output of the multimodal large model and determine whether the number of knowledge levels reaches the minimum threshold for the number of knowledge levels and whether the topic completeness meets the preset threshold for topic completeness. When the number of knowledge levels does not reach the minimum threshold for knowledge levels and the topic completeness is lower than the preset topic completeness threshold, a secondary parsing instruction will be automatically triggered to re-parse and extract the main line and knowledge structure of the video's global explanation. When the number of knowledge levels reaches the minimum threshold and the topic completeness meets the preset topic completeness threshold, the topic parsing is deemed valid and proceeds directly to the subsequent semantic encoding process.

[0009] In some embodiments, when inputting the filtered valid speech-text segments into a multimodal large model, parsing the global theme and knowledge level of the video, and performing semantic encoding on each speech segment and semantic association comparison of adjacent segments based on theme validity, clustering highly correlated segments into the same semantic unit according to a preset semantic matching logic, and defining the connection points of semantically different segments as presentation page boundaries, the method further includes: A semantic relevance matching threshold is pre-set, the semantic relevance values ​​of adjacent valid speech text segments are compared, and the overall context of the video is combined to determine whether the context before and after the text is consistent. When the semantic relevance of adjacent speech segments reaches the preset semantic relevance matching threshold and the context before and after the speech segments are consistent, they are determined to be the same knowledge topic and the adjacent segments are clustered and merged. When the semantic relevance of adjacent audio segments does not reach the preset semantic relevance matching threshold or there is a significant disconnect between the preceding and following contexts, they are determined to be different knowledge topics, and the presentation page boundaries are delineated at the connection between adjacent segments.

[0010] In some embodiments, the process of performing dual-dimensional compliance checks on each semantic unit based on video duration and text length, merging excessively short semantic units into adjacent units with the same topic, and performing secondary semantic decomposition of excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, includes: Pre-set reasonable ranges for video duration and text length for single-page presentations, and determine whether the video duration and text length of semantic units fall within the corresponding reasonable ranges. When either the video duration or the text length of a semantic unit deviates from the reasonable range, a joint review process for duration and text length is triggered. When both the video duration and text length of a semantic unit fall within a reasonable range, the semantic unit is initially deemed compliant and retained for the next pagination process.

[0011] In some embodiments, when performing dual-dimensional compliance checks on each semantic unit based on video duration and text length, merging excessively short semantic units into adjacent units with the same topic, and performing secondary semantic decomposition of excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, the method further includes: Determine whether the video duration or text length of a semantic unit is below the lower limit of a reasonable range or above the upper limit of a reasonable range, and simultaneously determine whether the topic affiliation of adjacent semantic units is consistent; When the duration or text length of a semantic unit is lower than the lower limit of a reasonable range and its topic is consistent with that of adjacent units, it is merged into the adjacent semantic units with the same topic. When the duration of a semantic unit or the length of a text exceeds the upper limit of a reasonable range, the multimodal large model will autonomously split the text into multiple independent pagination units according to the rule that knowledge points are independent and do not overlap.

[0012] In some embodiments, when performing dual-dimensional compliance checks on each semantic unit based on video duration and text length, merging excessively short semantic units into adjacent units with the same topic, and performing secondary semantic decomposition of excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, the method further includes: Establish a preset industry knowledge point keyword library, set a complete keyword coverage threshold, match the core points of the generated page with the preset industry knowledge point keyword library, and determine whether the keyword coverage of the core points reaches the complete keyword coverage threshold. When the coverage of core keywords does not reach the complete keyword coverage threshold, it is determined that there are missing key knowledge points, and the page content will be automatically supplemented, refined and improved. When the coverage of core keywords reaches the complete keyword coverage threshold, the page is deemed to have complete key points, and the page title and core summary content are directly fixed.

[0013] In some embodiments, when performing dual-dimensional compliance checks on each semantic unit based on video duration and text length, merging excessively short semantic units into adjacent units with the same topic, and performing secondary semantic decomposition of excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, the method further includes: Identify the content attributes of the input video, match it against the system's preset scene classification library, and determine whether the video belongs to a preset standard scene category; When a video is identified as belonging to a preset standard scene category, the page threshold and logic judgment rules of the corresponding scene are adaptively matched. When a video is identified as not belonging to a preset standard scene category, the system's default pagination threshold and logical judgment rules will be used. Relying on speech recognition models, speech activity detection models, and multimodal large models, the system performs full-process logical judgments, and the output structured pagination data is directly adapted to presentation tools to complete page rendering and generation.

[0014] Compared with existing technologies, the beneficial effects of this invention are as follows: This application relies on a multimodal large model to achieve intelligent processing of the entire video-to-presentation process, abandoning the traditional fixed-rule mechanical segmentation mode. It replaces subjective and fuzzy judgment with multi-level threshold quantification logic, making pagination standards unified and standardized. Combined with voice activity detection to correct segmentation boundaries, it can accurately remove invalid tone fragments, reasonably divide semantic breakpoints, and merge continuous explanation segments. It uses a multimodal large model to analyze the global theme and knowledge level of the video, generating semantic units and pagination boundaries through semantic association comparison and clustering. This application sets up a dual-dimensional compliance verification mechanism of duration and text, which can automatically complete the merging and secondary splitting of semantic units, supplement knowledge points with keyword library verification, and adaptively adjust pagination rules based on video scene types. The entire process requires no manual intervention, significantly reducing the time and labor costs of video-to-PPT conversion. The generated presentation page structure fits the video explanation logic, with complete and clear content hierarchy, adaptable to various scenarios such as knowledge sharing and education and training. The degree of automation, generation quality, and scene adaptability are significantly improved.

[0015] The above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0016] Other features and aspects of this disclosure will become clearer from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 The flowchart illustrates the automatic generation method for video-to-presentation conversion driven by a multimodal large model, as provided in this embodiment of the invention. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0021] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0022] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0023] See Figure 1As shown, a method for automatically generating video-to-presentation driven by a multimodal large model according to an embodiment of this application includes the following steps: S100: Obtain the target video network link and complete the video file download and format parsing; perform machine speech recognition and transcription on the entire video audio; and output the raw audio text coarse segmentation data with timestamp information. S200: The speech activity detection model is used to correct the start and end time boundaries of the coarse segmentation of the original speech text. Invalid speech segments are filtered out in combination with the text character length. Then, temporary semantic breakpoints are divided according to the time interval between adjacent speech segments and continuous semantic segments are merged. S300. Input the selected valid speech and text segments into the multimodal large model, analyze the global theme and knowledge level of the video, and perform semantic encoding on each speech segment and semantic association comparison of adjacent segments through theme validity. Based on the preset semantic matching logic, cluster highly related segments into the same semantic unit, and delineate the connection points of semantically different segments as the page boundaries of the presentation. S400 performs compliance checks on each semantic unit in two dimensions: video duration and text length. For semantic units that are too short, they are merged into adjacent units with the same theme. For semantic units that are too long, they are split into sub-knowledge points by a multimodal large model. Based on the compliant semantic units, the presentation automatically generates the title, key points and standardized structured pagination data for each page.

[0024] It should be understood that the entire system relies on the collaborative efforts of computer equipment, backend servers, a multimodal large-scale model inference engine, a speech recognition engine, and a speech activity detection engine. The working principle is as follows: using internet video URLs as the digital data source, the system first retrieves the digital video file, parses and preprocesses the encapsulated format, and performs transcoding. Then, it performs ASR speech recognition on the embedded digital audio signal in the video, converting it into structured electronic digital text data with timestamps. Through speech activity detection models such as SiloVAD, it performs fine-grained correction on the start and end timestamps of the audio, filters out invalid noise segments according to character length rules from the digital text dimension, and uses time interval temporal features to divide semantic breakpoints. Then, leveraging the semantic understanding capabilities of multimodal large-scale models such as GLM, it performs semantic vector encoding, contextual relevance calculation, and topic hierarchical deconstruction on the text segments, clustering them into semantic units and defining pagination boundaries. Finally, it completes compliance verification using duration and text length as dual threshold constraints, intelligently merging and splitting semantic units, and ultimately outputting standardized structured digital data that can be directly rendered into PowerPoint presentations. All logical judgments, threshold comparisons, and semantic calculations in this solution are automatically completed by the computer backend, requiring no manual intervention.

[0025] For example, when a Bilibili tutorial video URL is input, the system automatically downloads the MP4 video and parses the encapsulation format. It then uses the Whisper model to transcribe the entire speech and outputs coarsely segmented text with second-level timestamps. The VAD model corrects the start and end times of each speech segment, eliminating invalid segments with too few words or excessive redundancy. Semantic breakpoints are defined based on segment intervals, and continuous explanations are merged. After being fed into a multimodal large model, the system automatically organizes the overall course theme and chapter knowledge levels, performs semantic encoding on each text segment, automatically clusters adjacent texts of the same knowledge point into semantic units, and sets PPT page boundaries at knowledge point transitions. Finally, compliance checks are performed based on preset single-page video lengths of 30-120 seconds and text length thresholds of 150-600 words, automatically merging excessively short units and splitting excessively long units. Ultimately, the system automatically generates the title, summary points, and structured text data for each page.

[0026] In some specific embodiments, the step of obtaining the target video network link and completing the video file download and format parsing, performing machine speech recognition and transcription on the entire video audio, and outputting coarsely segmented raw audio-text data carrying timestamp information includes: The system identifies video container format types in a preset priority order and determines whether the current video format is a system preset standard playback format. When the current video format is not the system's default standard playback format, the system will automatically perform format transcoding and adaptation processing to convert it to the standard video format before proceeding with the subsequent speech recognition and transcription operation. When the current video format is the system's preset standard playback format, the speech recognition and transcription process will proceed directly.

[0027] It should be understood that this step is a crucial part of video digital data preprocessing. The preset priority order is the format recognition sorting that the system has pre-fixed, prioritizing the recognition of the mainstream MP4 format, followed by non-standard container formats such as FLV, AVI, and MKV. The system's preset standard playback format is uniformly set to the MP4 universal encoding format to ensure the compatibility of the subsequent speech recognition engine and frame extraction module. Format transcoding adaptation is completed by the backend built-in transcoding program, which only adjusts the video container and encoding format without changing the original video content and audio timing, ensuring that the subsequent speech transcription timestamp is accurate and without offset. The working principle is to read the container encoding information in the video file header, match and compare it with the system's preset format library, and execute the direct process or transcoding process according to the matching result branch. This belongs to the automatic identification and normalization preprocessing of digital file formats.

[0028] For example, if the input video is in MKV non-standard container format, the system will identify and determine that it does not belong to the preset MP4 standard format according to priority, and automatically start the transcoding program to transcode it losslessly into the standard MP4 format; if the input video itself is in the MP4 standard format, the transcoding process will be skipped and the video will be sent directly to the speech recognition module.

[0029] By employing a pre-defined priority format recognition mechanism, the system achieves automatic compatibility and adaptation to various network video encapsulation formats, avoiding issues such as speech recognition failure, timestamp errors, and video parsing anomalies caused by non-standard formats. It unifies and normalizes to a standard video format, providing a stable input source for subsequent VAD detection, speech-to-text transcription, and keyframe extraction. Fully automatic format identification and transcoding eliminates the need for manual format conversion, improving the overall automation level of the process. Prioritized recognition reduces invalid format traversal, improving video parsing efficiency. It also avoids system compatibility issues caused by differences in video encoding across different platforms, enhancing platform adaptability and robustness.

[0030] In some specific embodiments, the step of using a speech activity detection model to correct the start and end time boundaries of the coarse segmentation of the original speech text, filtering out invalid speech segments based on the text character length, and then dividing temporary semantic breakpoints and merging continuous semantic segments according to the time interval between adjacent speech segments includes: Determine whether the number of characters in a single audio text segment meets the system's preset character standard and whether the proportion of redundant text tone exceeds the preset ratio; When the number of characters in a single audio-text segment does not reach the preset character standard and the proportion of redundant tone components exceeds the preset ratio, the segment is determined to be a meaningless tone fragment and is directly filtered out. When the number of characters in a single audio-text segment reaches the preset character standard and the proportion of redundant tone components does not exceed the preset ratio, it is determined to be a valid audio segment and retained for subsequent processing.

[0031] It should be understood that the system's preset character standard is the minimum effective character threshold for a single segment of speech text, which can be configured to 20 characters. Redundant elements in tone include verbal particles, repeated interjections, and meaningless interjections. The system's built-in redundant vocabulary library automatically calculates the threshold for the proportion of redundant words in a single segment of text. This step uses a dual logic judgment of character quantity threshold + redundancy ratio threshold. The working principle is to perform word segmentation statistics and redundant word matching ratio calculation on each segment of speech text with timestamps. If both conditions are met simultaneously, it is judged as an invalid fragment. If either condition is not met, a valid segment is retained, realizing intelligent cleaning and preprocessing of electronic digital text data.

[0032] For example, if a transcribed text has only 12 characters and modal particles account for more than 35%, and is below the preset standard of 20 characters, the system will determine it as a modal fragment and remove it directly; if a text has 38 characters and redundancy accounts for only 12%, it meets the dual threshold requirements and is determined as a valid fragment to be retained for subsequent semantic segmentation.

[0033] By using dual thresholds to accurately filter out meaningless verbal tics, pauses, interruptions, and short, useless fragments from videos, the interference of invalid data on subsequent semantic pagination is reduced; the original voice and text data is purified, reducing the computational load of large-scale semantic parsing; invalid fragments are prevented from forming separate PPT fragment pages, ensuring that each page is rich and valuable; standardized threshold discrimination replaces manual subjective screening, with unified and replicable screening standards; and the accuracy of subsequent semantic clustering and pagination boundary division is improved, making PPT content more concise, professional, and free of redundant nonsense.

[0034] In some specific embodiments, when the step of using a speech activity detection model to correct the start and end time boundaries of the coarse segmentation of the original speech text, filtering out invalid speech segments based on the text character length, and then dividing temporary semantic breakpoints and merging continuous semantic segments according to the time interval between adjacent speech segments, the step further includes: Determine whether the time interval between adjacent valid speech segments exceeds the system's preset interval range, and simultaneously determine whether the speech pause attribute is a short pause for natural explanation or a long pause for knowledge point switching; When the time interval between adjacent valid speech segments exceeds the preset range and is determined to be a long pause in knowledge point switching, then that point is marked as a formal semantic breakpoint. When the time interval between adjacent valid audio segments is within a preset range and is determined to be a natural short pause in the explanation, it is determined to be continuous explanation content, and the adjacent audio segments are temporarily merged into a whole segment.

[0035] It should be understood that the system's preset interval range is a configurable time interval threshold, with 0.5 seconds as the dividing line. The speech pause attribute is determined by the VAD model detecting the silence duration and the semantic association features before and after the pause. Short pauses are pauses for breathing during normal narration by the lecturer, while long pauses are deliberate pauses for switching chapters or knowledge points. The working principle is based on the digital video time sequence timestamp, calculating the time difference between the beginning and end of adjacent speech segments, and classifying pause attributes by combining the VAD silence detection results. Semantic breakpoints and continuous segments are divided by the dual conditions of time threshold and pause type, realizing intelligent segmentation that combines time and semantics.

[0036] For example, if two adjacent audio segments are 0.8 seconds apart and are detected as long pauses for switching knowledge points, the system marks them as natural PPT pagination semantic breakpoints; if adjacent segments are 0.3 seconds apart and are short pauses for breathing during normal explanations, the system automatically merges them into a continuous semantic segment.

[0037] Breaking away from the limitations of traditional methods that rely solely on fixed time intervals for mechanical segmentation, this approach integrates temporal thresholds and semantic attributes for pause discrimination, making semantic breakpoint segmentation more aligned with human presentation habits. It accurately distinguishes between breathing pauses and knowledge point switching pauses, preventing the misinterpretation of short, normal presentation pauses as multiple PPT slides. It automatically merges continuous presentation segments and marks knowledge switching breakpoints, providing a precise temporal foundation for defining pagination boundaries in subsequent large-scale models. This reduces pagination fragmentation and overly detailed page splitting, improving the regularity of PPT page structure. Fully automated temporal and pause discrimination eliminates the need for manual breakpoint marking, enhancing processing efficiency and intelligence.

[0038] In some specific embodiments, the step of inputting the filtered valid speech-text segments into a multimodal large model, parsing the global theme and knowledge level of the video, semantically encoding each speech segment based on theme validity, comparing the semantic associations of adjacent segments, clustering highly correlated segments into the same semantic unit according to a preset semantic matching logic, and defining the connection points of semantically different segments as the page boundaries of the presentation, includes: Pre-set a minimum threshold for the number of knowledge levels and a threshold for topic completeness. Count the number of knowledge levels in the initial parsing output of the multimodal large model and determine whether the number of knowledge levels reaches the minimum threshold for the number of knowledge levels and whether the topic completeness meets the preset threshold for topic completeness. When the number of knowledge levels does not reach the minimum threshold for knowledge levels and the topic completeness is lower than the preset topic completeness threshold, a secondary parsing instruction will be automatically triggered to re-parse and extract the main line and knowledge structure of the video's global explanation. When the number of knowledge levels reaches the minimum threshold and the topic completeness meets the preset topic completeness threshold, the topic parsing is deemed valid and proceeds directly to the subsequent semantic encoding process.

[0039] It should be understood that the minimum number of knowledge levels is preset to at least 3 levels, including the main topic, chapters, and sub-knowledge points; the topic completeness threshold is a quantified score of the information completeness of the topic description output by the large model, which is quantified by semantic information content and keyword coverage; the working principle is that after the large model outputs the topic structure for the first time, the computer automatically counts the number of levels and calculates the topic completeness score, compares it with the preset dual thresholds, and if it does not meet the requirements, a second Prompt instruction is issued to allow the large model to reconstruct the topic, and if it meets the requirements, it is allowed to enter semantic encoding, thus avoiding the problems of ambiguous topic parsing and chaotic levels from the source.

[0040] For example, if the large model initially outputs only level 2 knowledge and the topic completeness score is lower than the preset 60 points, the system will automatically trigger a second analysis to reorganize the overall explanation of the video and the multi-level knowledge. If it outputs level 3 or above and the completeness score is higher than 60 points, the analysis will be directly determined to be valid.

[0041] By replacing the original fuzzy subjective judgment with quantitative dual-threshold constraints, the problems of vague topic descriptions, missing knowledge levels, and chaotic structures are completely solved. A secondary parsing retry mechanism is set up to significantly improve the accuracy and completeness of the global topic and knowledge level parsing of the video. A stable and reliable top-level knowledge framework is provided for subsequent semantic clustering and pagination. It avoids PPT pagination logic errors and chapter confusion caused by topic parsing deviations. Standardized threshold judgment makes the topic parsing standards of different videos consistent, and the system's stability and generalization ability are stronger. It reduces the workload of subsequent manual correction of topic structure and further improves the level of automation of the whole process.

[0042] In some specific embodiments, when inputting the filtered valid speech-text segments into a multimodal large model, parsing the global theme and knowledge level of the video, and performing semantic encoding on each speech segment and semantic association comparison of adjacent segments based on theme validity, clustering highly correlated segments into the same semantic unit according to a preset semantic matching logic, and defining the connection points of semantically different segments as the page boundaries of the presentation, the method further includes: A semantic relevance matching threshold is pre-set, the semantic relevance values ​​of adjacent valid speech text segments are compared, and the overall context of the video is combined to determine whether the context before and after the text is consistent. When the semantic relevance of adjacent speech segments reaches the preset semantic relevance matching threshold and the context before and after the speech segments are consistent, they are determined to be the same knowledge topic and the adjacent segments are clustered and merged. When the semantic relevance of adjacent audio segments does not reach the preset semantic relevance matching threshold or there is a significant disconnect between the preceding and following contexts, they are determined to be different knowledge topics, and the presentation page boundaries are delineated at the connection between adjacent segments.

[0043] It should be understood that the semantic relevance matching threshold is a quantitative threshold based on the cosine similarity of semantic vectors, with a preset fixed benchmark value; the semantic relevance is quantified by vector distance calculation after the large model generates semantic vectors for each text segment; the global context adaptation uses the overall theme of the video and the knowledge hierarchy framework as references to determine whether the narrative logic of adjacent segments is coherent; the working principle is a dual judgment of semantic vector numerical comparison + global context logic verification, which realizes automatic clustering of segments with the same theme and automatic generation of pagination boundaries for positions with different themes.

[0044] For example, if the semantic similarity between two adjacent audio text segments reaches a preset threshold and the context revolves around explaining the same sub-knowledge point, the system automatically clusters and merges them into a semantic unit; if the similarity is low and the context jumps to a completely new knowledge point, then a PPT page boundary is set between the two segments.

[0045] By combining semantic vector quantization thresholding with global contextual verification, pagination boundaries are more aligned with knowledge logic compared to fixed-duration segmentation. It accurately clusters content on the same topic, preventing the same knowledge point from being split across multiple PPT pages. It identifies the jump locations of knowledge points with fragmented contexts, accurately generating pagination boundaries and ensuring clear PPT chapter divisions. Relying on large-scale model semantic vector calculations, it achieves automatic machine semantic understanding, eliminating the need for manual content logic analysis. It adapts to teaching videos with conversational explanations, interspersed examples, and complex knowledge point jumps, demonstrating stronger scenario adaptability. It significantly reduces the fragmentation and logical breaks in PPT content, improving the overall quality of presentations.

[0046] In some specific embodiments, the process of performing dual-dimensional compliance checks on each semantic unit based on video duration and text length, merging excessively short semantic units into adjacent units with the same topic, and performing secondary semantic splitting of excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, includes: Pre-set reasonable ranges for video duration and text length for single-page presentations, and determine whether the video duration and text length of semantic units fall within the corresponding reasonable ranges. When either the video duration or the text length of a semantic unit deviates from the reasonable range, a joint review process for duration and text length is triggered. When both the video duration and text length of a semantic unit fall within a reasonable range, the semantic unit is initially deemed compliant and retained for the next pagination process.

[0047] It should be understood that the reasonable range for video duration is preset to 30 to 120 seconds, and the reasonable range for text length is preset to 150 to 600 words, both of which can be configured according to the scenario. The linkage review process refers to the system automatically backtracking the semantic units that deviate from the range and associating them with the themes and semantic relevance of the semantic units before and after, providing a basis for subsequent merging or splitting. The working principle is to establish dual threshold constraints from the time dimension and the text content dimension, and simultaneously check the volume of semantic units. If either dimension exceeds the standard, a review will be initiated. If both dimensions are qualified, the initial compliance will be determined, realizing standardized volume control of digital data.

[0048] For example, if a semantic unit corresponds to a video duration of 135 seconds, which exceeds the 120-second limit, a linked review is triggered; if a semantic unit has a duration of 55 seconds and text of 320 characters, both of which fall within the preset range, the initial compliance is directly determined.

[0049] Establish standardized threshold ranges for both duration and text, unifying the page size standard for all video-generated PPTs to prevent overly simplistic or overly complex page content. Exceeding any dimension triggers a linked review, providing data support for subsequent intelligent merging and splitting. This avoids arbitrary page division based on manual experience, resulting in well-formatted and balanced PPTs. The standardized ranges are adaptable to common scenarios such as education and training, and technical sharing, with flexible threshold configuration to suit different industry needs. Dual-dimensional verification significantly reduces the probability of unqualified semantic units entering subsequent pagination processes, improving the final PPT page's neatness and reading experience.

[0050] In some specific embodiments, the process of performing dual-dimensional compliance checks on each semantic unit based on video duration and text length, merging excessively short semantic units into adjacent units with the same topic, and performing secondary semantic splitting of excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, further includes: Determine whether the video duration or text length of a semantic unit is below the lower limit of a reasonable range or above the upper limit of a reasonable range, and simultaneously determine whether the topic affiliation of adjacent semantic units is consistent; When the duration or text length of a semantic unit is lower than the lower limit of a reasonable range and its topic is consistent with that of adjacent units, it is merged into the adjacent semantic units with the same topic. When the duration of a semantic unit or the length of a text exceeds the upper limit of a reasonable range, the multimodal large model will autonomously split the text into multiple independent pagination units according to the rule that knowledge points are independent and do not overlap.

[0051] It should be understood that the consistency of topic affiliation is determined by semantic matching of the large model to confirm whether adjacent units belong to the same knowledge chapter; the rule of independent and non-overlapping knowledge points requires that each sub-unit after splitting contains complete sub-knowledge points and there is no content overlap or redundancy between them; the working principle is to classify long and short units based on upper and lower thresholds, short units are merged according to topic consistency, and long units are decomposed into semantic sub-knowledge points by the large model to achieve intelligent regularization of semantic units.

[0052] For example, if a semantic unit is only 25 seconds long, which is below the lower limit, and belongs to the same chapter theme as the previous unit, the system will automatically merge it into a complete unit; if a unit is 150 seconds long, which exceeds the upper limit, the large model will automatically split it into three independent pagination units according to the three sub-knowledge points inside.

[0053] For short semantic units, intelligent merging of units within the same theme is implemented to avoid generating thin, blank pages. For long semantic units, precise splitting by sub-knowledge points is achieved to avoid bloated and complex content on single pages. Strict adherence to the rule of independent and non-overlapping knowledge points ensures that the split pages are logically complete and the content is not redundant. Based on the theme consistency constraint, only units within the same theme are merged to prevent logical confusion caused by forced merging across knowledge points. Fully automatic merging and splitting eliminates the need for manual adjustment of page content size, further enhancing the automation of the entire process. The standardized semantic unit structure better conforms to presentation production standards, significantly improving readability and display effects.

[0054] In some specific embodiments, the process of performing dual-dimensional compliance checks on each semantic unit based on video duration and text length, merging excessively short semantic units into adjacent units with the same topic, and performing secondary semantic splitting of excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, further includes: Establish a preset industry knowledge point keyword library, set a complete keyword coverage threshold, match the core points of the generated page with the preset industry knowledge point keyword library, and determine whether the keyword coverage of the core points reaches the complete keyword coverage threshold. When the coverage of core keywords does not reach the complete keyword coverage threshold, it is determined that there are missing key knowledge points, and the page content will be automatically supplemented, refined and improved. When the coverage of core keywords reaches the complete keyword coverage threshold, the page is deemed to have complete key points, and the page title and core summary content are directly fixed.

[0055] It should be understood that the industry knowledge point keyword database is pre-configured according to fields such as education, technology, and training, storing core professional terms and knowledge point keywords in the field; the keyword coverage completeness threshold is a quantitative percentage standard to determine the completeness of the generated summary containing core knowledge points; the working principle is to use keyword matching statistics to calculate the coverage score, compare it with the threshold to automatically identify missing knowledge points, and then use a multimodal big model to supplement the missing knowledge points based on the original text, so as to realize the intelligent verification and improvement of the page's key points.

[0056] For example, if a technical sharing page is missing three professional keywords in its core points, and the coverage does not reach the preset threshold, the system will automatically call the keyword database for comparison and use the large model to supplement the missing knowledge points and improve the summary content; if the keywords are fully covered, the existing title and key points will be directly retained.

[0057] By leveraging an industry keyword database and coverage threshold verification mechanism, the system automatically detects and fills in missing core knowledge points on pages, preventing the generation of PPTs from omitting key professional content. Standardized keyword constraints enhance the professionalism and rigor of presentations. The large model automatically supplements content without manual editing, saving on post-production layout and modification work. The industry-specific keyword database can be flexibly expanded to adapt to the video-to-PPT needs of different professional fields. This ensures that the core points on each page are accurate, complete, and professional, significantly enhancing the knowledge value and educational value of the generated PPTs.

[0058] In some specific embodiments, the process of performing dual-dimensional compliance checks on each semantic unit based on video duration and text length, merging excessively short semantic units into adjacent units with the same topic, and performing secondary semantic splitting of excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, further includes: Identify the content attributes of the input video, match it against the system's preset scene classification library, and determine whether the video belongs to a preset standard scene category; When a video is identified as belonging to a preset standard scene category, the page threshold and logic judgment rules of the corresponding scene are adaptively matched. When a video is identified as not belonging to a preset standard scene category, the system's default pagination threshold and logical judgment rules will be used. Relying on speech recognition models, speech activity detection models, and multimodal large models, the system performs full-process logical judgments, and the output structured pagination data is directly adapted to presentation tools to complete page rendering and generation.

[0059] It should be understood that the system's preset scene classification library includes standard scene types such as knowledge sharing, vocational training, course teaching, and technology popularization; the pagination threshold includes a full set of parameters such as character threshold, time interval threshold, semantic relevance threshold, and duration text interval threshold; adaptive matching refers to the system automatically loading the corresponding threshold parameters and logical judgment rules according to the scene category; the working principle is to complete scene classification through video title, voice text keywords, and theme features, and switch the entire set of thresholds and discrimination rules according to the classification results to achieve scene adaptive intelligent pagination control.

[0060] For example, if a video is identified as a course instruction video, the system automatically loads a relaxed duration range and a high knowledge point completeness threshold specifically for teaching scenarios; if it is identified as a general science popularization video, the system's default threshold and judgment rules are used to complete the pagination.

[0061] This system achieves automatic video scene recognition and adaptive switching of a full set of threshold rules, overcoming the drawback of a single fixed threshold being unable to adapt to multiple types of videos. It matches exclusive pagination logic and parameters to different scenes, enabling teaching, training, and science popularization videos to generate PPT pages tailored to their specific characteristics. It eliminates the need for manual adjustment of threshold parameters for different videos, significantly enhancing intelligent adaptation capabilities. Multi-model collaborative linkage ensures stable and reliable logic judgment throughout the entire process. The output structured pagination data has strong universality and can seamlessly adapt to automatic rendering in various presentation software. It greatly improves the system's generalization ability, scene adaptability, and batch processing capabilities, making it suitable for large-scale automatic generation of presentations from multiple industries and video types.

[0062] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for automatically generating video-to-presentation conversion driven by a multimodal large model, characterized in that, Includes the following steps: Obtain the target video network link and complete the video file download and format parsing; perform machine speech recognition and transcription on the entire video audio; and output coarse segmentation data of the original audio text with timestamp information. The speech activity detection model is used to correct the start and end time boundaries of the coarse segmentation of the original speech text. Invalid speech segments are filtered out by combining the text character length. Then, temporary semantic breakpoints are divided according to the time interval between adjacent speech segments and continuous semantic segments are merged. The selected valid speech and text segments are input into a multimodal large model to analyze the global theme and knowledge level of the video. The semantics of each speech segment are semantically encoded and the semantic association of adjacent segments is compared based on the theme validity. According to the preset semantic matching logic, highly related segments are clustered into the same semantic unit, and the connection points of semantically different segments are defined as the page boundaries of the presentation. Each semantic unit undergoes compliance verification based on both video duration and text length. Excessively short semantic units are merged into adjacent units with the same theme, while excessively long semantic units are further semantically split by a multimodal large model according to sub-knowledge points. Based on compliant semantic units, the presentation automatically generates page titles, key points, and standardized structured pagination data for each page.

2. The method for automatically generating video-to-presentation conversion driven by a multimodal large model according to claim 1, characterized in that, The process of obtaining the target video network link and completing video file download and format parsing, performing machine speech recognition and transcription on the entire video audio, and outputting coarsely segmented raw audio-text data with timestamp information includes: The system identifies video container format types in a preset priority order and determines whether the current video format is a system preset standard playback format. When the current video format is not the system's default standard playback format, the system will automatically perform format transcoding and adaptation processing to convert it to the standard video format before proceeding with the subsequent speech recognition and transcription operation. When the current video format is the system's preset standard playback format, the speech recognition and transcription process will proceed directly.

3. The method for automatically generating video-to-presentation conversion driven by a multimodal large model according to claim 2, characterized in that, The process of using a speech activity detection model to correct the start and end time boundaries of the coarse segmentation of the original speech text, filtering out invalid speech segments based on the text character length, and then dividing temporary semantic breakpoints and merging continuous semantic segments according to the time interval between adjacent speech segments includes: Determine whether the number of characters in a single audio text segment meets the system's preset character standard and whether the proportion of redundant text tone exceeds the preset ratio; When the number of characters in a single audio-text segment does not reach the preset character standard and the proportion of redundant tone components exceeds the preset ratio, the segment is determined to be a meaningless tone fragment and is directly filtered out. When the number of characters in a single audio-text segment reaches the preset character standard and the proportion of redundant tone components does not exceed the preset ratio, it is determined to be a valid audio segment and retained for subsequent processing.

4. The method for automatically generating video-to-presentation conversion driven by a multimodal large model according to claim 3, characterized in that, The process of using a speech activity detection model to correct the start and end time boundaries of the coarse segmentation of the original speech text, filtering out invalid speech segments based on the text character length, and then dividing temporary semantic breakpoints and merging continuous semantic segments according to the time interval between adjacent speech segments, further includes: Determine whether the time interval between adjacent valid speech segments exceeds the system's preset interval range, and simultaneously determine whether the speech pause attribute is a short pause for natural explanation or a long pause for knowledge point switching; When the time interval between adjacent valid speech segments exceeds the preset range and is determined to be a long pause in knowledge point switching, then that point is marked as a formal semantic breakpoint. When the time interval between adjacent valid audio segments is within a preset range and is determined to be a natural short pause in the explanation, it is determined to be continuous explanation content, and the adjacent audio segments are temporarily merged into a whole segment.

5. The method for automatically generating video-to-presentation conversion driven by a multimodal large model according to claim 4, characterized in that, The process of inputting the filtered valid speech-text segments into a multimodal large model, parsing the global theme and knowledge level of the video, semantically encoding each speech segment based on theme validity, comparing the semantic associations of adjacent segments, clustering highly correlated segments into the same semantic unit according to a preset semantic matching logic, and defining the connection points of semantically different segments as the page boundaries of the presentation includes: Pre-set a minimum threshold for the number of knowledge levels and a threshold for topic completeness. Count the number of knowledge levels in the initial parsing output of the multimodal large model and determine whether the number of knowledge levels reaches the minimum threshold for the number of knowledge levels and whether the topic completeness meets the preset threshold for topic completeness. When the number of knowledge levels does not reach the minimum threshold for knowledge levels and the topic completeness is lower than the preset topic completeness threshold, a secondary parsing instruction will be automatically triggered to re-parse and extract the main line and knowledge structure of the video's global explanation. When the number of knowledge levels reaches the minimum threshold and the topic completeness meets the preset topic completeness threshold, the topic parsing is deemed valid and proceeds directly to the subsequent semantic encoding process.

6. The method for automatically generating video-to-presentation conversion driven by a multimodal large model according to claim 5, characterized in that, The process of inputting the filtered valid speech-text segments into a multimodal large model, analyzing the global theme and knowledge level of the video, semantically encoding each speech segment based on theme validity, comparing the semantic associations of adjacent segments, clustering highly correlated segments into the same semantic unit according to a preset semantic matching logic, and defining the connection points of semantically different segments as the page boundaries of the presentation, further includes: A semantic relevance matching threshold is pre-set, the semantic relevance values ​​of adjacent valid speech text segments are compared, and the overall context of the video is combined to determine whether the context before and after the text is consistent. When the semantic relevance of adjacent speech segments reaches the preset semantic relevance matching threshold and the context before and after the speech segments are consistent, they are determined to be the same knowledge topic and the adjacent segments are clustered and merged. When the semantic relevance of adjacent audio segments does not reach the preset semantic relevance matching threshold or there is a significant disconnect between the preceding and following contexts, they are determined to be different knowledge topics, and the presentation page boundaries are delineated at the connection between adjacent segments.

7. The method for automatically generating video-to-presentation conversion driven by a multimodal large model according to claim 6, characterized in that, The process of performing dual-dimensional compliance checks on each semantic unit based on video duration and text length, merging excessively short semantic units into adjacent units with the same topic, and further semantically splitting excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, includes: Pre-set reasonable ranges for video duration and text length for single-page presentations, and determine whether the video duration and text length of semantic units fall within the corresponding reasonable ranges. When either the video duration or the text length of a semantic unit deviates from the reasonable range, a joint review process for duration and text length is triggered. When both the video duration and text length of a semantic unit fall within a reasonable range, the semantic unit is initially deemed compliant and retained for the next pagination process.

8. The method for automatically generating video-to-presentation conversion driven by a multimodal large model according to claim 7, characterized in that, The process of performing dual-dimensional compliance checks on video duration and text length for each semantic unit, merging excessively short semantic units into adjacent units with the same topic, and performing secondary semantic decomposition of excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, also includes: Determine whether the video duration or text length of a semantic unit is below the lower limit of a reasonable range or above the upper limit of a reasonable range, and simultaneously determine whether the topic affiliation of adjacent semantic units is consistent; When the duration or text length of a semantic unit is lower than the lower limit of a reasonable range and its topic is consistent with that of adjacent units, it is merged into the adjacent semantic units with the same topic. When the duration of a semantic unit or the length of a text exceeds the upper limit of a reasonable range, the multimodal large model will autonomously split the text into multiple independent pagination units according to the rule that knowledge points are independent and do not overlap.

9. The method for automatically generating video-to-presentation conversion driven by a multimodal large model according to claim 8, characterized in that, The process of performing dual-dimensional compliance checks on video duration and text length for each semantic unit, merging excessively short semantic units into adjacent units with the same topic, and performing secondary semantic decomposition of excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, also includes: Establish a preset industry knowledge point keyword library, set a complete keyword coverage threshold, match the core points of the generated page with the preset industry knowledge point keyword library, and determine whether the keyword coverage of the core points reaches the complete keyword coverage threshold. When the coverage of core keywords does not reach the complete keyword coverage threshold, it is determined that there are missing key knowledge points, and the page content will be automatically supplemented, refined and improved. When the coverage of core keywords reaches the complete keyword coverage threshold, the page is deemed to have complete key points, and the page title and core summary content are directly fixed.

10. The method for automatically generating video-to-presentation conversion driven by a multimodal large model according to claim 9, characterized in that, The process of performing dual-dimensional compliance checks on video duration and text length for each semantic unit, merging excessively short semantic units into adjacent units with the same topic, and performing secondary semantic decomposition of excessively long semantic units by a multimodal large model according to sub-knowledge points, and automatically generating the presentation's page title, core points, and standardized structured pagination data based on compliant semantic units, also includes: Identify the content attributes of the input video, match it against the system's preset scene classification library, and determine whether the video belongs to a preset standard scene category; When a video is identified as belonging to a preset standard scene category, the page threshold and logic judgment rules of the corresponding scene are adaptively matched. When a video is identified as not belonging to a preset standard scene category, the system's default pagination threshold and logical judgment rules will be used. Relying on speech recognition models, speech activity detection models, and multimodal large models, the system performs full-process logical judgments, and the output structured pagination data is directly adapted to presentation tools to complete page rendering and generation.

Images