A video fission method, device, equipment and storage medium

Abstract

The application discloses a video fission method, device, equipment and storage medium. First, two types of materials, a mother video and a script text, can be acquired, multi-modal structured analysis is performed on the mother video to obtain a video fingerprint sequence which is continuous in time sequence, and semantic splitting is performed on the script text to obtain a time sequence semantic key sequence composed of script keys. Then, scene unit multi-modal features and script key text semantic features are extracted respectively and encoded into two types of feature vectors. Then, the similarity between each script semantic vector and each scene feature vector is calculated to obtain a similarity matrix, and a video script mapping table is constructed based on the similarity matrix using a matching calculation algorithm. Finally, based on the video script mapping table, a video segment corresponding to a matching scene unit is cut from the mother video and sequentially synthesized to generate a target clip video. The application can not only reduce the artificial cost required for video slicing creation, but also accelerate the film production speed and improve the accuracy of script and picture matching.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence and multimedia video processing technology, specifically to a video splitting method, apparatus, device, and storage medium. Background Technology

[0002] With the rise of short-video social platforms, content creators and marketing teams need to re-edit the same long video content to produce multiple short videos with different focuses to maximize the dissemination effect. This process is usually called "video fission" or "slice distribution". In the current video fission workflow, creators need to use professional editing software to repeatedly browse the master video, rely on their own memory and subjective judgment to select video scenes that match the target topic, cut the master video into segments, select suitable content from a large number of video segments and reassemble them as needed. At the same time, in order to adapt to the content tone of different platforms, it is also necessary to rethink the script and produce matching voice-overs for each video segment. This operation process is cumbersome and time-consuming, and it requires a high level of content understanding and rhythm control from the editors. Furthermore, when adapting to different platform styles, it is necessary to rewrite the script and generate matching voice-overs for each segment separately. As a result, the secondary creation of video slices has the drawbacks of high manual operation costs and low content production efficiency.

[0003] Therefore, how to reduce the manual operation costs in the secondary creation process of videos and improve the efficiency and accuracy of the process is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a video splitting method, apparatus, device, and storage medium that can automate the splitting of long videos into segments, reduce manual operation costs, and improve the efficiency and standardization of secondary video creation.

[0005] The embodiments of this application disclose the following technical solutions: A video splitting method, the method comprising: Obtain the master video and script text to be processed; The master video is subjected to multimodal structured parsing to obtain a temporally continuous video fingerprint sequence, and the script text is subjected to semantic decomposition to obtain a temporally continuous semantic point sequence; the video fingerprint sequence includes multiple scene units, each scene unit is labeled with corresponding audio text, time interval and visual label; the semantic point sequence includes multiple script points; Multimodal feature encoding is performed on the voice text and visual labels annotated in each scene unit of the video fingerprint sequence to obtain scene feature vectors, and text semantic encoding is performed on each script point in the semantic point sequence to obtain script semantic vectors. A similarity matrix is ​​obtained by calculating the similarity between the semantic vectors of each script and the feature vectors of each scene, and a video script mapping table is constructed based on the similarity matrix using a matching calculation algorithm. The video script mapping table is used to describe the temporal matching relationship between each script key point and the corresponding scene unit. The matching calculation algorithm includes a dynamic time warping algorithm or an improved Hungarian algorithm that carries temporal monotonic constraints and supports single key point matching of consecutive adjacent segments. Based on the video script mapping table, video segments corresponding to matching scene units are extracted from the master video and synthesized in an orderly manner to generate the target edited video.

[0006] In one possible implementation, the step of performing multimodal structured parsing on the parent video to obtain a temporally continuous video fingerprint sequence includes: The master video is processed by a convolutional neural network (CNN) to extract frame features and perform transition segmentation to obtain multiple visually continuous video segments. Visual content recognition is then performed on each video segment to obtain the corresponding visual tags. Each video segment is bound with a start and end time. Speech recognition is performed on the master video audio track to obtain a subtitle text sequence; the subtitle text sequence includes multiple text segments, which are segmented based on punctuation marks, and each text segment is bound to a corresponding time interval; For each text segment in the subtitle text sequence, one or more video clips that have temporal overlap and visual continuity with the time interval bound to that text segment are aggregated to obtain a scene unit; For each scene unit, mark the time interval covered by the scene unit as well as one or more visual labels corresponding to the scene unit. All scene units labeled with visual tags and time intervals are sorted and stored according to video time sequence to obtain the video fingerprint sequence.

[0007] In one possible implementation, the parent video is processed using a CNN to extract frame features and perform transition segmentation to obtain multiple visually continuous video segments, including: The master video is used to extract features by CNN to obtain multiple single-frame visual feature vectors; By comparing the visual feature vectors of the multiple single frames, the next frame in the adjacent frames in which the feature difference between the next frame and the previous frame exceeds the difference threshold is determined as the transition boundary frame. The parent video is segmented based on all transition frames to obtain the multiple visually continuous video segments.

[0008] In one possible implementation, the semantic segmentation of the script text to obtain a temporally continuous sequence of semantic key points includes: The script text is semantically boundary identified by a text semantic extraction model to obtain at least one semantic turning point. The script text is segmented based on all semantic inflection points to obtain multiple script key points; The multiple script key points are sorted and stored based on the original chronological order of the text to obtain the semantic key point sequence.

[0009] In one possible implementation, constructing the video script mapping table based on the similarity matrix using a matching algorithm includes: Based on the matching calculation algorithm and the similarity matrix, the matching relationship between script key points and scene units is globally optimized and multiple sets of script scene matching entries are obtained. The multiple sets of script scene matching entries are structured and archived to obtain the video script mapping table.

[0010] In one possible implementation, the step of extracting video segments corresponding to matching scene units from the parent video based on the video script mapping table and performing ordered synthesis to generate the target edited video includes: The video script mapping table is traversed according to the order of the script points in the semantic point sequence to obtain the scene unit group corresponding to each script point; the scene unit group includes one or more scene units. For each scene unit group, based on the time interval marked for each scene unit within the scene unit group, the original video segment corresponding to the scene unit group is extracted from the master video; Create a blank video track, and arrange and splice all the original video clips on the blank video track according to the order of the script points in the semantic point sequence to obtain the basic video material track; All script key points are converted from text to speech to obtain initial audio. Based on the total duration of the basic video material track, the initial audio is adaptively matched and adjusted to obtain dubbing audio that is adapted to the basic video material track. Obtain auxiliary editing materials, and synchronously overlay the auxiliary editing materials and the dubbing audio onto the basic video material track to obtain the target edited video; the auxiliary editing materials include background music, transition effects and / or brand watermarks.

[0011] A video splitting device, the device comprising: The material acquisition unit is used to acquire the master video and script text to be processed. A multimodal parsing unit is used to perform multimodal structured parsing on the master video to obtain a temporally continuous video fingerprint sequence; the video fingerprint sequence includes multiple scene units, each scene unit is labeled with corresponding speech text, time interval and visual labels; The script semantic segmentation unit is used to perform semantic segmentation on the script text to obtain a temporally continuous sequence of semantic key points; the sequence of semantic key points includes multiple script key points. The feature encoding unit is used to perform multimodal feature encoding on the speech text and visual labels annotated by each scene unit in the video fingerprint sequence to obtain scene feature vectors, and to perform text semantic encoding on each script point in the semantic point sequence to obtain script semantic vectors. The similarity calculation unit is used to calculate the similarity between each script semantic vector and each scene feature vector to obtain a similarity matrix; A temporal matching unit is used to construct a video script mapping table based on the similarity matrix using a matching calculation algorithm. The video script mapping table is used to describe the temporal matching relationship between each script point and the corresponding scene unit. The matching calculation algorithm includes a dynamic time warping algorithm or an improved Hungarian algorithm that carries temporal monotonic constraints and supports single point matching of consecutive adjacent segments. The video synthesis unit is used to extract video segments corresponding to matching scene units from the master video based on the video script mapping table and synthesize them in an orderly manner to generate the target edited video.

[0012] In one possible implementation, the device further includes: The video parsing unit, used for performing multimodal structured parsing on the parent video to obtain a temporally continuous video fingerprint sequence, includes: The shot segmentation unit is used to extract frame features and perform transition segmentation processing on the master video using CNN to obtain multiple visually continuous video segments; each video segment is bound to a start and end time. The visual recognition unit is used to perform visual content recognition on each video segment to obtain the visual tags corresponding to each video segment. An audio transcription unit is used to perform speech recognition on the master video audio track to obtain a subtitle text sequence; the subtitle text sequence includes multiple text segments, which are segmented based on punctuation marks, and each text segment is bound to a corresponding time interval; The segment aggregation unit is used to aggregate one or more video segments that have temporal overlap and visual continuity with the time interval bound to the text segment for each text segment in the subtitle text sequence, so as to obtain a scene unit; Unit annotation unit is used to annotate each scene unit with the time interval covered by the entire scene unit and one or more visual labels corresponding to the scene unit. The first sorting and storage unit is used to sort and store all scene units labeled with visual tags and time intervals according to the video time sequence to obtain the video fingerprint sequence.

[0013] A video splitting device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the video splitting method described above.

[0014] A computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the video splitting method as described above.

[0015] Compared with the prior art, this application has the following beneficial effects: This application provides a video splitting method, apparatus, device, and storage medium. Specifically, when executing the video splitting method provided in this application embodiment, the parent video to be processed and the accompanying script text can be obtained first, providing complete original materials and creative basis for subsequent fully automated editing. Then, the parent video is subjected to multimodal structured analysis to generate a temporally continuous video fingerprint sequence. Each scene unit in this sequence is labeled with corresponding audio text, time interval, and visual tags. Simultaneously, the script text is semantically decomposed to obtain a semantic point sequence composed of multiple ordered script points. This step transforms unstructured long videos and plain text scripts into standardized, quantifiable temporal data, completely eliminating the need for manual repeated video browsing and subjective image selection. Next, multimodal feature encoding is performed on the audio text and visual tags labeled for each scene unit in the video fingerprint sequence to generate scene feature vectors that accurately represent video scene features. Simultaneously, text semantic encoding is performed on each script point in the semantic point sequence to generate script semantic vectors that reflect the core semantics of the script. This enables cross-modal numerical comparison between video audiovisual content and script text, effectively avoiding image matching deviations caused by differences in human understanding. Next, the similarity between the semantic vectors of each script and the feature vectors of each scene is calculated, and a corresponding similarity matrix is ​​constructed. Based on this similarity matrix, a matching algorithm with temporal monotonic constraints, capable of matching a single script point with multiple consecutive adjacent segments, is used to complete the matching operation. A video script mapping table is established to record the temporal matching relationship between each script point and the corresponding scene unit. This ensures that the matching order strictly follows the script's narrative logic and adapts to real-world editing scenarios where a single script segment corresponds to multiple consecutive frames. Finally, based on this video script mapping table, the video segments corresponding to the scene units in the parent video that match each script point are accurately located. The extracted video segments are then combined in an orderly manner according to the chronological order of the script points to generate the final target edited video. This completely replaces all repetitive manual cutting, dragging, and recombining of materials, simplifying the traditionally cumbersome editing process. The entire method significantly reduces manual intervention through the orderly connection of each step. This application performs multimodal structured parsing on the parent video and semantic decomposition on the script text, producing scene units and script key points with complete temporal sequence and multidimensional annotations. Then, through feature encoding, similarity calculation, and a matching algorithm with temporal constraints, it automatically establishes a corresponding mapping between script content and video frames. This replaces the traditional process of repeatedly browsing the parent video and relying on subjective judgment to select, cut, and splice materials. It eliminates the rigid requirements for editors' professional interpretation and image control capabilities, simplifies the complicated manual slicing operation, reduces labor input costs, accelerates the video slicing output rate, and improves the accuracy of matching script semantics with video scenes. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a video splitting method provided in this application embodiment; Figure 2 A flowchart illustrating a video fingerprint sequence construction method provided in this application embodiment; Figure 3 A flowchart illustrating a method for generating edited videos provided in this application embodiment; Figure 4 This is a schematic diagram of the structure of a video splitting device provided in an embodiment of this application. Detailed Implementation

[0018] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0019] With the rapid development of short-video social platforms, video splitting (also known as video segment distribution) has become an important means for content creators and marketing teams to improve dissemination effectiveness. The core of this method is to re-edit the same long video master clip into multiple short videos with different focuses. Currently, this process relies entirely on manual operation. Creators need to repeatedly browse the master video using professional editing software, selecting and matching scenes to the target topic based on memory and subjective judgment, completing the segment cutting, filtering, and recombining. They also need to adapt to the tone of different platforms, conceiving individual scripts and producing matching voiceovers for each video segment. This model is cumbersome, time-consuming, and requires high levels of content understanding and pacing control from editors. Furthermore, it incurs high labor costs, ultimately resulting in low production efficiency and high costs for the secondary creation of video segments, failing to meet the actual need for efficient, large-scale production of short videos.

[0020] To address this issue, this application provides a video splitting method, apparatus, device, and storage medium. First, the parent video to be processed and its accompanying script text are acquired. Multimodal structured parsing is performed on the parent video to generate a temporally continuous video fingerprint sequence containing scene units carrying voice text, time intervals, and visual labels. Simultaneously, the script text is semantically split to obtain a semantic point sequence composed of multiple ordered script points. Features of both sequences are then extracted. Multimodal feature encoding is performed on the voice text and visual labels labeled for each scene unit to generate scene feature vectors. Text semantic encoding is performed on each script point to generate a script semantic vector. Subsequently, the similarity between all script semantic vectors and scene feature vectors is calculated pairwise to construct a corresponding similarity matrix. Based on this matrix, a matching algorithm with temporal monotonic constraints and capable of matching multiple consecutive adjacent segments with a single script point is used to complete the matching operation, establishing a video script mapping table to record the temporal matching association between each script point and its corresponding scene unit. Finally, based on this mapping table, the video segments in the parent video corresponding to each matched scene unit are located, and the extracted video segments are combined in temporal order to generate the target edited video. This application can reduce the labor costs required for video segment creation, speed up the production of finished products, and improve the accuracy of script and screen matching.

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0022] See Figure 1 The figure is a flowchart of a video splitting method provided in an embodiment of this application, as shown below. Figure 1 As shown, the video fission method may include steps S101-S105: S101: Obtain the master video and script text to be processed.

[0023] To complete the video splitting and editing work based on script-based automatic matching of images, two types of core raw materials can be obtained first: the complete master video to be processed and the accompanying script text. The master video contains all the original images, audio and video content information, and is the data source for subsequent cropping and generating short video slices. The script text records the theme description, dialogue content and narrative logic corresponding to each short video slice, and serves as a reference for matching and filtering video images.

[0024] S102: Perform multimodal structured parsing on the parent video to obtain a temporally continuous video fingerprint sequence, and perform semantic decomposition on the script text to obtain a temporally continuous semantic key point sequence.

[0025] To overcome the shortcomings of unstructured and unquantifiable comparisons of raw master videos and plain text scripts, and to eliminate the need for manual review and subjective selection of footage, a multimodal structured analysis of the master video and semantic decomposition of the script text can be performed separately. The complete master video is decomposed into a temporally continuous video fingerprint sequence, denoted as V1, V2, ..., Vn. This sequence is further divided into multiple independent scene units, each of which is synchronously bound to the corresponding segment's audio-visual text, precise time interval, and visual tags, thus preserving complete audiovisual and temporal information, denoted as [Script Key Points_P1, Script Key Points_P2, ...]. Simultaneously, the complete script text is decomposed according to narrative logic to form a temporally coherent semantic key point sequence. Multiple independent script key points constitute standardized text units. This transforms the two heterogeneous materials—audio-visual materials and text scripts—into standardized structured data with temporal and multi-dimensional annotation information, providing a regular and computable data foundation for subsequent cross-modal feature encoding and similarity matching operations.

[0026] S103: Multimodal feature encoding is performed on the voice text and visual labels labeled in each scene unit in the video fingerprint sequence to obtain scene feature vectors, and text semantic encoding is performed on each script point in the semantic point sequence to obtain script semantic vectors.

[0027] To achieve cross-modal quantitative comparison between video scene information and script text semantics, and to eliminate matching bias caused by subjective human judgment, this step simultaneously performs multimodal feature encoding on the audio text and visual tags attached to each scene unit in the video fingerprint sequence. This integrates the visual content of the video screen with the audio dialogue text and transforms it into a scene feature vector of a unified dimension, fully quantifying the comprehensive audiovisual features of a single shot. At the same time, text semantic encoding is performed separately on each script key point in the semantic key point sequence to extract the core semantics of the text and generate a standardized script semantic vector. In this way, all three types of information—image, audio, and text—are transformed into computable numerical vectors.

[0028] S104: Calculate the similarity between each script semantic vector and each scene feature vector to obtain a similarity matrix, and construct a video script mapping table based on the similarity matrix using a matching calculation algorithm.

[0029] To leverage the previously generated scene feature vectors and script semantic vectors to achieve an objective, accurate, and narrative-logical match between script key points and video scene units, avoiding issues such as disordered matching, omissions, and inconsistent matching that easily occur in traditional manual matching, and replacing the tedious manual comparison and binding of images and text line by line, this step can perform pairwise similarity calculations on all script semantic vectors and scene feature vectors. By quantifying the numerical values, it can intuitively reflect the degree of fit between each set of script key points and scene units, thereby constructing a similarity matrix that can comprehensively reflect the matching relationship between the two, providing clear data support for subsequent matching operations.

[0030] Based on this, a video script mapping table is constructed using the similarity matrix as the core basis and a matching calculation algorithm. This mapping table can clearly describe the temporal relationship between each script point and the corresponding matching scene unit, providing clear guidance for subsequent video segment extraction and synthesis.

[0031] The matching algorithm includes a dynamic time warping algorithm or an improved Hungarian algorithm that carries temporal monotonic constraints and supports single-point matching of consecutive adjacent segments. Specifically, both the dynamic time warping algorithm and the improved Hungarian algorithm selected in this application are equipped with targeted constraint rules. First, the temporal monotonic constraint requires the matching process to strictly follow the original chronological order of the script points, prohibiting out-of-order matching and ensuring that the final video slices conform to the narrative logic of the script text, avoiding problems such as chaotic transitions between shots and text, and logical gaps. Second, the single-point matching rule allows a single script point to match multiple temporally adjacent scene units, adapting to the creative needs of real-world editing scenarios where a single piece of text needs to connect multiple sets of coherent shots. Based on this matching method, the matching operation can be completed through global algorithm optimization, ensuring both the objectivity and accuracy of the matching results and significantly improving matching efficiency, effectively solving the technical shortcomings of manual matching, such as being time-consuming, error-prone, and having poor adaptability.

[0032] S105: Based on the video script mapping table, extract the video segments corresponding to the matching scene units from the master video and synthesize them in an orderly manner to generate the target edited video.

[0033] To translate the pre-matched script points and video scene units obtained through precise algorithmic matching into directly usable short video segments, replacing the tedious manual positioning, cropping, and dragging / recombining of video clips in traditional manual editing, and ensuring the final video content is coherent, highlights key points, and strictly adheres to the narrative logic of the script, this step can be based on a pre-constructed video script mapping table. This mapping table clearly records the scene unit corresponding to each script point, the precise time interval of each scene unit in the parent video, and the order of the script points, providing a clear and standardized execution basis for video segment extraction and synthesis. Based on the time intervals recorded in the mapping table, the video segment corresponding to each matching scene unit can be automatically and accurately located in the parent video, eliminating the need for manual timeline dragging and frame-by-frame searching for target images, and avoiding the subjective judgment of editors in determining the cropping range. This effectively avoids problems such as incomplete image capture and time positioning errors that easily occur with manual cropping. After all matching video segments have been extracted, the extracted video segments can be spliced ​​together in an orderly manner according to the original order of the script points. At the same time, the video and audio of each segment are automatically connected to ensure that the transition between segments is natural and without any stutters or gaps, and finally generate the target edited video that meets the expectations.

[0034] The entire process of synthesizing a complete short video according to the script logic requires no manual intervention. This not only completely simplifies the tedious manual editing process in traditional video splitting, but also ensures the content coherence and narrative logic of the target video. At the same time, it greatly improves the production efficiency of short video segments and effectively reduces manual operation costs and editing errors.

[0035] Based on the content of S101-S105, the process first involves acquiring the parent video and script text to be processed. Multimodal structured parsing is then performed on the parent video to generate a temporally continuous video fingerprint sequence containing multi-dimensionally labeled scene units. Semantic decomposition of the script text generates a temporally sequential semantic point sequence composed of script key points. Next, the speech text and visual labels of the scene units are extracted and multimodal feature encoding is performed to obtain scene feature vectors. Text semantic encoding is then performed on each script key point to obtain script semantic vectors. Subsequently, the similarity between the two types of vectors is calculated to construct a similarity matrix. Based on the similarity matrix and a matching algorithm with temporal monotonic constraints that supports single-point matching of consecutive adjacent segments, a video script mapping table is constructed to record the temporal matching relationship between script key points and scene units. Finally, video segments corresponding to scene units are extracted from the parent video using this mapping table and synthesized in an orderly manner to obtain the target edited video. This application directly simplifies the cumbersome manual process of traditional video splitting, reducing labor costs and simultaneously accelerating the segment generation speed, thereby improving the matching accuracy between scripts and video frames.

[0036] See Figure 2 , Figure 2 This application provides a flowchart of a video fingerprint sequence construction method. Accordingly, step S102, which involves performing multimodal structured parsing on the parent video to obtain a temporally continuous video fingerprint sequence, can be specifically implemented through steps S201-S205. S201: The master video is processed by CNN to extract frame features and perform transition segmentation to obtain multiple visually continuous video segments, and visual content recognition is performed on each video segment to obtain the visual label corresponding to each video segment.

[0037] To accurately segment the visual content of the master video, extract quantifiable image features, and provide standardized visual foundation data for subsequent scene unit aggregation and script image matching, eliminating the inefficient manual segmentation and content labeling of video clips, this step employs a Convolutional Neural Network (CNN) for professional processing of the master video. First, the CNN extracts the visual features of each frame of the master video. Then, based on the differences in frame features, transition segmentation is performed, breaking down the complete master video into multiple visually continuous video clips. Each clip is bound to a corresponding start and end time, clearly defining its temporal position within the master video and avoiding problems such as chaotic clip segmentation and ambiguous temporal positioning. After completing the video clip segmentation, visual content recognition is further performed on each visually continuous video clip. This recognition process encompasses three core stages: object detection, scene recognition, and face recognition. By accurately identifying the core objects, scene type, and characters within the clip, corresponding visual labels are generated for each video clip, such as ["Speaker A - Medium Shot", "Product B - Close-up", "Chart", "Audience - Full View"]. This quantifies the core content of the clip's image.

[0038] S202: Perform speech recognition on the master video audio track to obtain a subtitle text sequence.

[0039] To extract the narration semantic information carried by the master video audio track, establish a precise temporal correspondence between audio text and video images, and achieve automatic aggregation of subsequent audio text segments and visual video segments, this step performs speech recognition processing on the master video audio track, abandoning the inefficient operation of manually listening to audio sentence by sentence, manually recording lines and marking time intervals. The output is a standardized subtitle text sequence, uniformly denoted as T_audio(time,text). The recognized complete text is automatically segmented according to the punctuation marks in the sentences, splitting into multiple logically complete and semantically independent text segments. At the same time, each text segment is bound to its precise time interval fully covered in the video, clearly defining the playback period corresponding to each line of narration.

[0040] This method transforms continuous human voices in audio into structured text units with temporal markers, fully preserving the audio narrative semantics and temporal coordinates of the video. It can perform temporal overlap matching with visual video segments with start and end times obtained from previous steps, providing audio-side data support for the aggregation into complete scene units that combine visual information and narration text.

[0041] S203: For each text segment in the subtitle text sequence, aggregate one or more video clips that have temporal overlap and visual continuity with the time interval bound to that text segment to obtain a scene unit.

[0042] To achieve temporal fusion of the decomposed atomic-level visual segments with precisely timestamped audio text, this step integrates scattered, single-dimensional visual and dialogue units into independent analysis units with complete audiovisual semantics. This avoids the problem of fragmented visual and audio information and the inability to jointly match the script, while also eliminating the repetitive operation of manually binding shots and corresponding dialogues to the timeline. For each text segment in the subtitle text sequence, this step retrieves all single or multiple atomic-level video segments that have temporal overlap with the time interval of that text segment and maintain visual continuity, and aggregates them to generate complete scene units. This aggregation logic relies on T_audio's built-in timestamp and punctuation division rules to automatically group the scattered atomic visual segments V1, V2, etc. into scene units with complete narrative semantics.

[0043] For example, assuming the presenter's complete explanation of a product feature introduction line covers three consecutive shots (V3, V4, and V5) within a timeframe, the three visual segments can be automatically bound together to form scene unit _U1. After global temporal matching and aggregation processing, the original unstructured complete master video is transformed into a standardized structured video fingerprint library. Each scene unit in the library will fully include its corresponding consecutive visual segment sequence, matched audio text, overall coverage time interval, and all visual tags generated by each shot. This achieves deep binding of visual information and audio narration semantics at the temporal and content levels, allowing each independent scene to have complete and quantifiable multimodal description information. There is no longer any fragmented data where the visuals and dialogue are separated. This not only eliminates the temporal misalignment and semantic fragmentation defects caused by manual audio-visual pairing, but also provides a unified, complete, and semantically closed-loop basic data unit for subsequent multimodal feature encoding, script semantics, and global similarity matching calculations of video scenes, significantly improving the accuracy and computational efficiency of subsequent cross-modal matching.

[0044] S204: For each scene unit, mark the time interval covered by the scene unit as well as one or more visual labels corresponding to the scene unit on the scene unit.

[0045] To ensure that each aggregated scene unit possesses complete and readable temporal information and semantic identifiers, facilitating subsequent feature extraction for cross-modal matching and avoiding data retrieval chaos caused by scattered storage of visual labels and time ranges, this step extracts the complete time interval covered by the merged visual segments of each aggregated scene unit. Simultaneously, it summarizes all visual labels corresponding to all visual segments within the scene unit and fully annotates and binds these two types of information to the corresponding scene unit entity. After annotation, any scene unit can directly query its own complete playback period and all screen content descriptions without needing to traverse the internal sub-segment summary information a second time. This simplifies the subsequent feature encoding data reading process, ensures centralized and unified storage of multimodal information, and provides complete structured identifier information for the quantitative comparison between scripts and scene units.

[0046] S205: Sort and store all scene units labeled with visual tags and time intervals according to video time sequence to obtain the video fingerprint sequence.

[0047] To standardize and organize all scene units with completed multimodal annotations, fully preserve the original temporal sequence logic of the parent video, avoid temporal disorder caused by disordered storage of scene units, and prevent shot reversal issues in matching operations, and to provide an ordered and traversable standardized data source for subsequent matching algorithms with temporal monotonic constraints, this step strictly sorts and stores all scene units with fully annotated time intervals and multiple sets of visual tags in the order of their appearance in the parent video, ultimately generating a temporally continuous and structurally regular video fingerprint sequence. After temporally sorted storage, the entire video fingerprint sequence completely replicates the narrative timeline of the original video, with the position of each scene unit corresponding one-to-one with its actual playback order. The algorithm can maintain the temporal logic naturally without additional timestamp comparisons during traversal, perfectly adapting to the temporal monotonic constraint rules in subsequent matching algorithms and preventing the binding of script key points with shot order disorder.

[0048] Steps S201-S205 enable dual-modal temporal alignment and structured processing of video vision and audio text. By using scene units as basic fingerprint units, the temporal information of screen tags and subtitles is integrated and generated in a temporal order to achieve integrated representation of video content semantics and temporal information, thereby improving fingerprint matching accuracy and scene positioning capabilities.

[0049] In one possible implementation, step S201 involves using a CNN to extract frame features and perform transition segmentation on the parent video to obtain multiple visually continuous video segments, including: Feature extraction from the master video is performed using a CNN to obtain multiple single-frame visual feature vectors. Then, by comparing these single-frame visual feature vectors, the next frame in a series of adjacent frames where the feature difference between the subsequent frame and the preceding frame exceeds a difference threshold is identified as the transition boundary frame. Finally, the master video is segmented based on all transition boundary frames to obtain the multiple visually continuous video segments.

[0050] Specifically, firstly, a CNN is used to analyze the master video frame by frame, extracting unique pixel and semantic information for each frame to generate a one-to-one single-frame visual feature vector. Then, the single-frame visual feature vectors of adjacent frames are compared pairwise, and the feature difference between consecutive frames is quantified. When the feature difference between a subsequent frame and its preceding frame exceeds a preset difference threshold, a significant change in the scene content is determined, and this subsequent frame is marked as a transition boundary frame. After traversing the entire video and identifying all transition boundary frames, all boundary frames are used as segmentation nodes to divide the entire master video. The scenes between two adjacent sets of boundary frames constitute a video segment with coherent content and no obvious visual jumps, thus enabling the batch output of multiple visually continuous independent video segments.

[0051] This method relies on objectively quantified feature values ​​to determine shot transitions, without depending on human visual judgment of scene changes. It can stably identify various transition effects such as fade-in, fade-out, and hard cut. The internal image style and shooting subject of each segmented video segment remain consistent, providing clear-boundary and complete basic visual sub-segments for subsequent visual tag recognition and audio-text temporal aggregation to generate scene units.

[0052] In one possible implementation, step S102 involves semantically decomposing the script text to obtain a temporally continuous sequence of semantic key points, including: The script text is semantically boundary-identified using a text semantic extraction model to obtain at least one semantic inflection point. Then, the script text is segmented based on all semantic inflection points to obtain multiple script key points. Finally, the multiple script key points are sorted and stored based on the original text order to obtain the semantic key point sequence.

[0053] Specifically, the text semantic extraction model is first invoked to fully parse the entire text, identifying semantic boundaries corresponding to topic shifts and content transitions within each segment, and locating and outputting at least one semantic turning point in the entire text. Then, using all identified semantic turning points as segmentation nodes, the original complete script text is truncated and split, yielding several semantically independent script points with a single theme. Finally, strictly adhering to the original script text's chronological order, all the split script points are sequentially sorted and centrally stored, ultimately forming a temporally continuous sequence of semantic points.

[0054] This solution relies on models to automatically identify semantic boundaries rather than simply using punctuation to crudely segment text. It can accurately distinguish the text content of different promotion themes and different explanation modules. Each segment of the script has an independent and complete expression logic, and the original narrative sequence is preserved throughout. There will be no situation where the order of key points is reversed. This provides logically regular and temporally ordered textual foundation data for subsequent script semantic encoding and temporal constraint matching with video scene units.

[0055] In one possible implementation, step S104, which involves constructing a video script mapping table based on the similarity matrix using a matching algorithm, includes: Based on the matching calculation algorithm and the similarity matrix, a global optimal filtering is performed on the matching relationship between script key points and scene units to obtain multiple sets of script scene matching entries. Then, these multiple sets of script scene matching entries are structured and archived to obtain the video script mapping table.

[0056] Specifically, the process begins by using a pre-calculated similarity matrix, which contains the quantified scores of all pairwise matches, as the numerical basis. A global optimization algorithm, incorporating temporal monotonic constraints and supporting single-point matching of multiple continuous scene units, is then employed. This algorithm comprehensively filters candidate matching relationships between all script points and scene units, considering multiple constraints such as temporal order, matching similarity, and segment continuity. This results in multiple sets of script scene matching entries that balance logical coherence and content suitability. All the filtered matching entries are then structurally and uniformly archived according to the original script temporal order, clearly recording the matching scene unit, matching confidence, scene temporal range, and other related information for each script point. Finally, a standardized and searchable video script mapping table is generated.

[0057] This method abandons the coarse matching mode of comparing each segment locally. It balances the matching accuracy of a single segment with the overall smoothness of the final product through global optimal filtering. The archived mapping table has a clear hierarchy and complete related information. The subsequent system can directly locate the video segment corresponding to each script based on the mapping table, providing a standardized query basis for automatically extracting and orderly synthesizing target edited videos.

[0058] See Figure 3 , Figure 3 This application provides a flowchart of a method for generating edited videos. Accordingly, step S105 involves extracting video segments corresponding to matching scene units from the parent video based on the video script mapping table and performing ordered synthesis to generate the target edited video. This can be specifically implemented through steps S301-S305. S301: Traverse the video script mapping table according to the order of the script points in the semantic point sequence, and obtain the scene unit group corresponding to each script point.

[0059] To strictly adhere to the original narrative logic of the script text, accurately acquire the video footage resources corresponding to each script point, and provide a clear and orderly execution basis for subsequent video segment extraction and orderly synthesis, avoiding problems such as mismatched script points and scene units, and disordered image retrieval, this step performs an ordered traversal of the video script mapping table according to the chronological order of the script points in the semantic point sequence. The video script mapping table, as a standardized index file generated by the initial matching operation, fully records the matching relationship between each script point and its corresponding scene unit. Traversing it in the original order of the semantic point sequence ensures that subsequent image retrieval and synthesis always conform to the narrative flow of the script text, preventing situations where shots are disconnected from the script's logic or the order is reversed. During the traversal, for each script point, all corresponding scene units are accurately extracted from the mapping table, and these scene units are integrated into a scene unit group. Each scene unit group can contain one or more scene units. This setting aligns with the needs of real video editing; a single script point can correspond to an independent scene unit or multiple consecutive adjacent scene units with related content.

[0060] S302: For each scene unit group, extract the original video segment corresponding to the scene unit group from the master video according to the time interval marked by each scene unit in the scene unit group.

[0061] To accurately acquire video footage corresponding to each script point, automating and standardizing video clip extraction, and avoiding issues such as time positioning errors, incomplete extraction, omissions, or mis-extractions that easily occur during manual video extraction, while ensuring a high degree of compatibility between the extracted video clips and the script points, providing high-quality original material for subsequent video splicing and compositing, this step performs precise extraction for each scene unit group. Each scene unit group is generated in the early stages based on script points and contains one or more scene units that match the content of the corresponding script points. Each scene unit has been marked with a precise time interval during the early multimodal structured analysis stage. This time interval clearly defines the start and end times of the scene unit in the parent video, providing a clear and quantifiable execution basis for automated extraction. During the actual extraction process, the time intervals marked for all scene units within each scene unit group can be read one by one. Using this time interval as the extraction range, the corresponding time period's content in the parent video can be accurately located, automatically completing the extraction of the original video clip. For a scene unit group containing multiple scene units, video clips corresponding to each scene unit can be extracted sequentially according to the chronological order of the scene units in the parent video, ensuring that the extracted clips can completely cover all the screen content required by the script's key points and fit the script's narrative logic.

[0062] S303: Create a blank video track. Arrange all the original video clips sequentially and splice them onto the blank video track according to the order of the script points in the semantic point sequence to obtain the basic video material track.

[0063] To integrate the previously extracted, fragmented video clips into a coherent and script-aligned foundation of visual material, avoiding issues such as disordered clip order, disjointed transitions, and missing shots that often occur during manual video splicing, and to provide a unified operational platform for subsequent dubbing adaptation and auxiliary material overlay, this step first creates a blank video track. This blank track serves as the foundation for splicing and integrating all the original video clips. It clearly records key information such as the arrangement order and duration of the video clips, facilitating subsequent fully automated editing. Then, following the original chronological order of the script's key points in the semantic sequence, all the extracted original video clips are dragged, arranged, and spliced ​​onto this blank video track, ultimately forming a complete foundational video material track. This splicing logic is entirely based on the script's narrative flow, ensuring a high degree of consistency between the presentation order of the video footage and the order of the script's explanation.

[0064] S304: Perform text-to-speech synthesis on all script key points to obtain initial audio, and perform adaptive duration matching adjustment on the initial audio based on the total duration of the basic video material track to obtain dubbing audio adapted to the basic video material track.

[0065] To transform script text into voice-over content that can be directly matched with video footage, and to address the issues of time-consuming, costly, and prone to audio-visual misalignment and inconsistent speech pace with visual rhythm caused by manual voice-over, this step first integrates all script key points into complete text content. Then, text-to-speech technology is used to synthesize initial audio, which fully reproduces the semantic content of the script text, ensuring consistency between the voice-over narration and the core information of the script. This eliminates the need for manual recording and post-production audio editing, significantly reducing the manpower and time costs of the voice-over process. Since the duration of the initial audio is generated based on the number of words in the script text and the default speech pace, it often deviates from the total duration of the basic video footage track. Directly superimposing these two audio tracks can result in audio-visual asynchrony, where the voice-over finishes playing before the video ends, or vice versa. Therefore, the initial audio needs to be adaptively adjusted based on the total duration of the basic video footage track. During the adjustment process, the system first accurately calculates the total duration of the basic video material track, then compares the difference between the initial audio duration and the total track duration. By intelligently adjusting the voice-over speed (appropriately speeding up or slowing down the speed while ensuring clear and intelligible speech and complete semantics), and reasonably trimming or adding silent segments, the system ensures that the total duration of the adjusted voice-over audio is completely consistent with the total duration of the basic video material track. At the same time, it ensures that the pauses and rhythms of the voice-over sentences match the playback rhythm of the picture, avoiding situations where the speech is too fast to be heard clearly or too slow to slow down the rhythm of the picture.

[0066] S305: Obtain auxiliary editing materials, and synchronously overlay the auxiliary editing materials and the dubbing audio onto the basic video material track to obtain the target edited video.

[0067] To perfect the final audiovisual presentation and unify the brand visual identity, while aligning and integrating voice-over audio, various auxiliary materials, and the main visuals in a consistent timeline, this step eliminates the tedious manual importing and segment-by-segment alignment of audio and video materials. Pre-selected auxiliary editing materials are retrieved in advance. The material types can be flexibly chosen from one or more combinations of background music, transition effects, and brand watermarks, depending on the editing needs. Then, a multi-track synchronous overlay operation is performed, uniformly mounting and overlaying the voice-over audio, adapted to the video length, and all auxiliary editing materials onto the already spliced ​​base video material track. Multiple layers of materials share the same timing benchmark, automatically completing timeline matching and alignment, ultimately outputting a complete target edited video ready for direct distribution.

[0068] The background music loops and adapts to the start and end times of the basic video track throughout the entire presentation, filling in blank periods and enhancing the overall atmosphere. Transition effects are automatically filled at the junctions of two original video segments, eliminating the abrupt and disjointed feel of camera cuts. The brand watermark is fixedly superimposed on a designated area of ​​the screen and displayed throughout the presentation, ensuring continuous exposure of brand information.

[0069] The entire overlay process automatically merges multiple layers of material based on a unified video timing sequence. This eliminates the need for editors to manually adjust audio start and end points, drag effects positions, or adjust watermark display durations. It ensures that the timing of voice-over and background music volumes does not conflict, and that transitions, watermarks, and the main visual are not misaligned. It also integrates and outputs visuals, voice-over, and packaging materials in one go, solving the shortcomings of traditional editing's cumbersome multi-track layering process and the tendency for multiple layers of material to misalign. While ensuring that the narrative logic of the finished product conforms to the original script, it simultaneously completes standardized audiovisual packaging, directly generating a finished short video that features complete narration, smooth shot transitions, brand logos, and background music.

[0070] Steps S301-S305 first traverse the mapping table according to the script key points to obtain each set of matching scene units. Based on the timestamps carried by the scene units, the corresponding original video segments are automatically extracted and spliced ​​in time to generate the basic video track. Then, based on all the script key points, standardized dubbing audio with the appropriate track length is generated. Finally, auxiliary editing materials such as background music, transition effects, and brand watermarks are integrated with the dubbing audio and superimposed on the basic track to complete the fully automated video slicing and synthesis process. There is no need for manual selection, dragging and recombining of materials, or separate dubbing production for each slice.

[0071] See Figure 4 , Figure 4 This is a schematic diagram of a video splitting device provided in an embodiment of this application. Figure 4 As shown, the video splitting device includes: Material acquisition unit 401 is used to acquire the master video to be processed and the script text; The multimodal parsing unit 402 is used to perform multimodal structured parsing on the master video to obtain a temporally continuous video fingerprint sequence; the video fingerprint sequence includes multiple scene units, and each scene unit is labeled with corresponding speech text, time interval and visual tags; The script semantic segmentation unit 403 is used to perform semantic segmentation on the script text to obtain a temporally continuous sequence of semantic key points; the sequence of semantic key points includes multiple script key points. The feature encoding unit 404 is used to perform multimodal feature encoding on the speech text and visual labels annotated by each scene unit in the video fingerprint sequence to obtain scene feature vectors, and to perform text semantic encoding on each script point in the semantic point sequence to obtain script semantic vectors. The similarity calculation unit 405 is used to calculate the similarity between each script semantic vector and each scene feature vector to obtain a similarity matrix; The temporal matching unit 406 is used to construct a video script mapping table based on the similarity matrix using a matching calculation algorithm; the video script mapping table is used to describe the temporal matching association between each script point and the corresponding scene unit; the matching calculation algorithm includes a dynamic time warping algorithm or an improved Hungarian algorithm that carries temporal monotonic constraints and supports single point matching of consecutive adjacent segments. The video synthesis unit 407 is used to extract video segments corresponding to matching scene units from the master video based on the video script mapping table and synthesize them in an orderly manner to generate the target edited video.

[0072] In one possible implementation, the device further includes: The video parsing unit, used for performing multimodal structured parsing on the parent video to obtain a temporally continuous video fingerprint sequence, includes: The shot segmentation unit is used to extract frame features and perform transition segmentation processing on the master video using CNN to obtain multiple visually continuous video segments; each video segment is bound to a start and end time. The visual recognition unit is used to perform visual content recognition on each video segment to obtain the visual tags corresponding to each video segment. An audio transcription unit is used to perform speech recognition on the master video audio track to obtain a subtitle text sequence; the subtitle text sequence includes multiple text segments, which are segmented based on punctuation marks, and each text segment is bound to a corresponding time interval; The segment aggregation unit is used to aggregate one or more video segments that have temporal overlap and visual continuity with the time interval bound to the text segment for each text segment in the subtitle text sequence, so as to obtain a scene unit; Unit annotation unit is used to annotate each scene unit with the time interval covered by the entire scene unit and one or more visual labels corresponding to the scene unit. The first sorting and storage unit is used to sort and store all scene units labeled with visual tags and time intervals according to the video time sequence to obtain the video fingerprint sequence.

[0073] In one possible implementation, the lens segmentation unit is specifically used for: The master video is used to extract features by CNN to obtain multiple single-frame visual feature vectors; By comparing the visual feature vectors of the multiple single frames, the next frame in the adjacent frames in which the feature difference between the next frame and the previous frame exceeds the difference threshold is determined as the transition boundary frame. The parent video is segmented based on all transition frames to obtain the multiple visually continuous video segments.

[0074] In one possible implementation, the script semantic splitting unit 403 is specifically used for: The script text is semantically boundary identified by a text semantic extraction model to obtain at least one semantic turning point. The script text is segmented based on all semantic inflection points to obtain multiple script key points; The multiple script key points are sorted and stored based on the original chronological order of the text to obtain the semantic key point sequence.

[0075] In one possible implementation, the timing matching unit 406 is specifically used for: Based on the matching calculation algorithm and the similarity matrix, the matching relationship between script key points and scene units is globally optimized and multiple sets of script scene matching entries are obtained. The multiple sets of script scene matching entries are structured and archived to obtain the video script mapping table.

[0076] In one possible implementation, the video synthesis unit 407 is specifically used for: The video script mapping table is traversed according to the order of the script points in the semantic point sequence to obtain the scene unit group corresponding to each script point; the scene unit group includes one or more scene units. For each scene unit group, based on the time interval marked for each scene unit within the scene unit group, the original video segment corresponding to the scene unit group is extracted from the master video; Create a blank video track, and arrange and splice all the original video clips on the blank video track according to the order of the script points in the semantic point sequence to obtain the basic video material track; All script key points are converted from text to speech to obtain initial audio. Based on the total duration of the basic video material track, the initial audio is adaptively matched and adjusted to obtain dubbing audio that is adapted to the basic video material track. Obtain auxiliary editing materials, and synchronously overlay the auxiliary editing materials and the dubbing audio onto the basic video material track to obtain the target edited video; the auxiliary editing materials include background music, transition effects and / or brand watermarks.

[0077] In addition, this application embodiment also provides a video splitting device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the video splitting method as described above.

[0078] In addition, this application embodiment also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the video splitting method described above.

[0079] This application's embodiments simultaneously introduce a multimodal structured parsing mechanism for the parent video and a semantic decomposition mechanism for the script text, generating scene units and script elements with complete temporal and multi-dimensional annotation information respectively. Then, by using feature encoding, similarity calculation, and a matching algorithm with temporal constraints, the mapping relationship between the script and the video frame is automatically established. This eliminates the need for manual review of the parent video and the extensive operation of subjectively selecting, cutting, and recombining segments. It also eliminates the reliance on the editor's own content interpretation and rhythm control capabilities, simplifying the traditional cumbersome manual splitting process, effectively reducing labor costs, and simultaneously improving the speed of segment generation and the accuracy of script-frame matching.

[0080] The foregoing has provided a detailed description of a video splitting method, apparatus, device, and storage medium provided in this application. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0081] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0082] It should also be noted that, in this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

Claims

1. A video fission method, characterized in that, The method includes: Obtain the master video and script text to be processed; The master video is subjected to multimodal structured parsing to obtain a temporally continuous video fingerprint sequence, and the script text is subjected to semantic decomposition to obtain a temporally continuous semantic point sequence; the video fingerprint sequence includes multiple scene units, each scene unit is labeled with corresponding audio text, time interval and visual label; the semantic point sequence includes multiple script points; Multimodal feature encoding is performed on the voice text and visual labels annotated in each scene unit of the video fingerprint sequence to obtain scene feature vectors, and text semantic encoding is performed on each script point in the semantic point sequence to obtain script semantic vectors. A similarity matrix is ​​obtained by calculating the similarity between the semantic vectors of each script and the feature vectors of each scene, and a video script mapping table is constructed based on the similarity matrix using a matching calculation algorithm. The video script mapping table is used to describe the temporal matching relationship between each script key point and the corresponding scene unit. The matching calculation algorithm includes a dynamic time warping algorithm or an improved Hungarian algorithm that carries temporal monotonic constraints and supports single key point matching of consecutive adjacent segments. Based on the video script mapping table, video segments corresponding to matching scene units are extracted from the master video and synthesized in an orderly manner to generate the target edited video.

2. The method according to claim 1, characterized in that, The step of performing multimodal structured parsing on the master video to obtain a temporally continuous video fingerprint sequence includes: The master video is processed by a convolutional neural network (CNN) to extract frame features and perform transition segmentation to obtain multiple visually continuous video segments. Visual content recognition is then performed on each video segment to obtain the corresponding visual tags. Each video segment is bound with a start and end time. Speech recognition is performed on the master video audio track to obtain a subtitle text sequence; the subtitle text sequence includes multiple text segments, which are segmented based on punctuation marks, and each text segment is bound to a corresponding time interval; For each text segment in the subtitle text sequence, one or more video clips that have temporal overlap and visual continuity with the time interval bound to that text segment are aggregated to obtain a scene unit; For each scene unit, mark the time interval covered by the scene unit as well as one or more visual labels corresponding to the scene unit. All scene units labeled with visual tags and time intervals are sorted and stored according to video time sequence to obtain the video fingerprint sequence.

3. The method according to claim 2, characterized in that, Multiple visually continuous video segments are obtained by performing frame feature extraction and transition segmentation processing on the master video using CNN, including: The master video is used to extract features by CNN to obtain multiple single-frame visual feature vectors; By comparing the visual feature vectors of the multiple single frames, the next frame in the adjacent frames in which the feature difference between the next frame and the previous frame exceeds the difference threshold is determined as the transition boundary frame. The parent video is segmented based on all transition frames to obtain the multiple visually continuous video segments.

4. The method according to claim 1, characterized in that, The semantic segmentation of the script text to obtain a temporally continuous sequence of semantic key points includes: The script text is semantically boundary identified by a text semantic extraction model to obtain at least one semantic turning point. The script text is segmented based on all semantic inflection points to obtain multiple script key points; The multiple script key points are sorted and stored based on the original chronological order of the text to obtain the semantic key point sequence.

5. The method according to claim 1, characterized in that, The step of constructing a video script mapping table based on the similarity matrix using a matching calculation algorithm includes: Based on the matching calculation algorithm and the similarity matrix, the matching relationship between script key points and scene units is globally optimized and multiple sets of script scene matching entries are obtained. The multiple sets of script scene matching entries are structured and archived to obtain the video script mapping table.

6. The method according to claim 1, characterized in that, The step of extracting video segments corresponding to matching scene units from the master video based on the video script mapping table and performing ordered synthesis to generate the target edited video includes: The video script mapping table is traversed according to the order of the script points in the semantic point sequence to obtain the scene unit group corresponding to each script point; the scene unit group includes one or more scene units. For each scene unit group, based on the time interval marked for each scene unit within the scene unit group, the original video segment corresponding to the scene unit group is extracted from the master video; Create a blank video track, and arrange and splice all the original video clips on the blank video track according to the order of the script points in the semantic point sequence to obtain the basic video material track; All script key points are converted from text to speech to obtain initial audio. Based on the total duration of the basic video material track, the initial audio is adaptively matched and adjusted to obtain dubbing audio that is adapted to the basic video material track. Obtain auxiliary editing materials, and synchronously overlay the auxiliary editing materials and the dubbing audio onto the basic video material track to obtain the target edited video; the auxiliary editing materials include background music, transition effects and / or brand watermarks.

7. A video splitting device, characterized in that, The device includes: The material acquisition unit is used to acquire the master video and script text to be processed. A multimodal parsing unit is used to perform multimodal structured parsing on the master video to obtain a temporally continuous video fingerprint sequence; the video fingerprint sequence includes multiple scene units, each scene unit is labeled with corresponding speech text, time interval and visual labels; The script semantic segmentation unit is used to perform semantic segmentation on the script text to obtain a temporally continuous sequence of semantic key points; the sequence of semantic key points includes multiple script key points. The feature encoding unit is used to perform multimodal feature encoding on the speech text and visual labels annotated by each scene unit in the video fingerprint sequence to obtain scene feature vectors, and to perform text semantic encoding on each script point in the semantic point sequence to obtain script semantic vectors. The similarity calculation unit is used to calculate the similarity between each script semantic vector and each scene feature vector to obtain a similarity matrix; A temporal matching unit is used to construct a video script mapping table based on the similarity matrix using a matching calculation algorithm. The video script mapping table is used to describe the temporal matching relationship between each script point and the corresponding scene unit. The matching calculation algorithm includes a dynamic time warping algorithm or an improved Hungarian algorithm that carries temporal monotonic constraints and supports single point matching of consecutive adjacent segments. The video synthesis unit is used to extract video segments corresponding to matching scene units from the master video based on the video script mapping table and synthesize them in an orderly manner to generate the target edited video.

8. The apparatus according to claim 7, characterized in that, The device further includes: The video parsing unit, used for performing multimodal structured parsing on the parent video to obtain a temporally continuous video fingerprint sequence, includes: The shot segmentation unit is used to extract frame features and perform transition segmentation processing on the master video using CNN to obtain multiple visually continuous video segments; each video segment is bound to a start and end time. The visual recognition unit is used to perform visual content recognition on each video segment to obtain the visual tags corresponding to each video segment. An audio transcription unit is used to perform speech recognition on the master video audio track to obtain a subtitle text sequence; the subtitle text sequence includes multiple text segments, which are segmented based on punctuation marks, and each text segment is bound to a corresponding time interval; The segment aggregation unit is used to aggregate one or more video segments that have temporal overlap and visual continuity with the time interval bound to the text segment for each text segment in the subtitle text sequence, so as to obtain a scene unit; Unit annotation unit is used to annotate each scene unit with the time interval covered by the entire scene unit and one or more visual labels corresponding to the scene unit. The first sorting and storage unit is used to sort and store all scene units labeled with visual tags and time intervals according to the video time sequence to obtain the video fingerprint sequence.

9. A video splitting device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the video splitting method as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the video splitting method as described in any one of claims 1-6.

Images