An AI short video intelligent splicing system of multi-source heterogeneous elements

Abstract

The application discloses a kind of multi-source heterogeneous material AI short video intelligent splicing systems, it is related to artificial intelligence video processing technical field, the system includes following component parts: material preprocessing module, heterogeneous feature extraction module, cross-modal label mapping module, label linkage matching module and material preselection pool module;The application guarantees standardization level by material preprocessing, feature extraction generates cross material correlation characteristics, solves material quality and feature representation problem;Call pre-training multimodal big model to realize cross-modal semantic accurate alignment and standardized label generation, material matching link multidimensional screening, correlation check, double guarantee material quality, improve material adaptation efficiency and splicing effect coherence and consistency, satisfy multiple high-quality creation needs.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence video processing technology, specifically to an AI short video intelligent splicing system for multi-source heterogeneous materials. Background Technology

[0002] With the rapid rise of the short video industry, video creation is increasingly demanding more diversified and efficient content, making the fusion of multi-source heterogeneous materials a mainstream creative model. Traditional short video splicing relies on manual selection, editing, and adaptation, which is not only time-consuming and labor-intensive but also struggles to guarantee semantic coherence and emotional consistency between materials. To improve creative efficiency, AI-assisted splicing technology has emerged. However, existing systems still have shortcomings in standardized processing of multi-source heterogeneous materials, cross-modal feature fusion, and accurate theme matching, failing to fully meet users' needs for high-quality, personalized short video creation. Therefore, an efficient and intelligent splicing solution is urgently needed.

[0003] Existing AI short video splicing technologies have many shortcomings when processing multi-source heterogeneous materials. Material preprocessing is superficial, merely converting formats without effectively removing redundant information such as black frames and watermarks. It also lacks calibration of parameters like color, volume, and character encoding, resulting in inconsistent quality of standardized materials. Feature extraction focuses on basic features of a single modality, neglecting semantic relationships, temporal correspondence, and emotional relevance between materials. Furthermore, feature fusion algorithms lack multi-dimensional collaborative consideration, making it difficult to form accurate fused feature vectors. Cross-modal label mapping lacks a dynamic calibration mechanism, resulting in low label standardization. Theme-material matching relies solely on single-dimensional label comparison, failing to establish a label-material-theme linkage model, leading to insufficient screening accuracy. In addition, the lack of a robust secondary verification and re-matching process easily leads to problems such as material adaptation conflicts and theme deviations.

[0004] The aforementioned shortcomings of existing technologies have resulted in problems such as low creation efficiency, poor material adaptability, and insufficient semantic coherence in AI short video splicing, hindering the intelligent development of the industry. Against the backdrop of increasingly diverse creative demands, the market urgently needs an intelligent system capable of standardized processing of multi-source heterogeneous materials, accurate fusion of cross-modal features, efficient theme matching, and rigorous quality verification. This invention addresses the core pain points of existing technologies by employing a multi-module collaborative design to achieve intelligent processing of materials throughout the entire process, effectively improving both creation efficiency and quality. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an AI-powered intelligent splicing system for multi-source heterogeneous materials. This system can comprehensively process multi-source heterogeneous materials through a well-established material preprocessing workflow, achieving format unification, redundancy removal, parameter calibration, and semantic verification to ensure material standardization. The feature extraction stage deeply mines the relationships between materials, generating cross-material related features, and outputs a fully fused feature vector through collaborative fusion. Simultaneously, it calls a pre-trained multimodal large model to parse semantics and combines dynamic label mapping to achieve accurate alignment and standardized label generation, improving material adaptation efficiency and the consistency of splicing effects.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: an AI short video intelligent splicing system for multi-source heterogeneous materials, the system comprising the following components: material preprocessing module, heterogeneous feature extraction module, cross-modal tag mapping module, tag linkage matching module, and material pre-selection pool module; The material preprocessing module collects four types of raw heterogeneous materials: video clips, images, audio, and text scripts, and performs format standardization, redundant information removal, parameter calibration, and semantic verification to output standardized heterogeneous materials. The heterogeneous feature extraction module: receives standardized heterogeneous materials, extracts the basic visual, auditory, and textual features of the standardized heterogeneous materials to form basic feature data; based on the basic feature data, analyzes the semantic, temporal, and emotional correlation between different materials through association analysis technology to generate cross-material association features; and uses a multi-source heterogeneous collaborative feature fusion algorithm to collaboratively fuse the basic feature data and the cross-material association features to output a fused feature vector. The cross-modal label mapping module: receives the fused feature vector, calls the pre-trained multimodal large model, parses the cross-modal semantic information in the fused feature vector; and uses the cross-modal dynamic label mapping calibration algorithm to align and calibrate the cross-modal semantic information to generate a standardized label set. The tag linkage matching module receives a standardized tag set and an externally input short video creation theme, breaks down the theme into a demand tag set and performs multi-dimensional matching with the standardized tag set to filter out highly compatible materials; establishes a tag-material-theme linkage model, and outputs candidate material combinations after verifying the highly compatible materials. The material pre-selection pool module receives candidate material combinations, stores them by type, sorts them according to matching degree, and performs secondary verification; it triggers a re-matching process for materials that fail verification, and finally generates an optimized material call list, which constitutes the candidate material pre-selection pool.

[0007] Furthermore, the process of outputting standardized heterogeneous materials in the material preprocessing module is as follows: video clips, images, and audio are collected through local storage and terminal device interfaces; text scripts are collected through database retrieval and creation platform interfaces; video clips are standardized in encoding format, resolution, and frame rate; images are standardized in image format and color gamut; audio is standardized in sampling rate; and text scripts are standardized in encoding format and text format; black frames, still frames, and invalid transition segments in video clips are removed; watermarks, blank edges, and repeated pixel blocks in images are removed; silent and noisy segments in audio are cleared; and repeated statements in text scripts are deleted; the color space, brightness, and contrast of the video are calibrated; the pixel density of images is adjusted; the audio volume is balanced; and the character encoding of text scripts is standardized; the semantic consistency between video images and embedded subtitles, image visual content and additional descriptions, and audio speech content and text scripts is verified; after verification, standardized heterogeneous materials are output.

[0008] Furthermore, the heterogeneous feature extraction module analyzes the semantic, temporal, and sentiment correlations between different materials using correlation analysis technology to generate cross-material correlation features. The process is as follows: extract the semantic content of each standardized heterogeneous material and establish semantic correspondences between heterogeneous materials; for materials with temporal attributes, mark time node information, match the time correspondence positions of non-temporal materials and temporal materials, and determine the temporal relationship between materials; identify the sentiment tendency features of each standardized heterogeneous material, compare the consistency of sentiment tendencies of different materials, and clarify the sentiment correlations between materials; integrate the correlation data of the three dimensions of semantics, temporality, and sentiment to generate cross-material correlation features.

[0009] Furthermore, the calculation formula for the multi-source heterogeneous collaborative feature fusion algorithm in the heterogeneous feature extraction module is as follows: ,in, This is the final output fused feature vector; The normalized weighting coefficients adjust the contribution ratio of different feature terms in the fusion result. It is a basic feature vector, which is composed of visual, auditory, and textual basic feature attributes extracted from standardized heterogeneous materials. This is a cross-material association feature vector that reflects the semantic, temporal, and emotional relationships between different heterogeneous materials. It integrates the basic feature data of standardized heterogeneous materials with cross-material association features, connects the semantic, temporal, and emotional relationship information between heterogeneous materials, eliminates the differences in feature dimensions between different types of materials, and forms a fusion feature vector that can comprehensively represent the comprehensive attributes of materials.

[0010] Furthermore, the pre-trained multimodal large model in the cross-modal label mapping module specifically includes a feature receiving layer, a cross-modal encoding layer, a feature fusion layer, and a semantic decoding layer. The feature receiving layer is used to receive the fused feature vector, split and adapt the receiving formats of visual, auditory, and textual modal features. The cross-modal encoding layer contains multiple cascaded encoding units, captures the semantic associations between features of different modalities and completes feature encoding. The feature fusion layer integrates the encoded multimodal features through attention weight allocation to eliminate semantic bias between modalities. The semantic decoding layer parses the cross-modal semantic information in the fused features and outputs structured semantic representation information.

[0011] Furthermore, the calculation formula for the cross-modal dynamic label mapping calibration algorithm in the cross-modal label mapping module is as follows: ,in, For a single calibrated label in the standardized label set, the output of the cross-modal dynamic label mapping calibration algorithm is given. The total number of modes participating in the calibration; For modal indexing; For the first The cross-modal semantic information representation value of each modality; For the first Dynamic weighting coefficients for each modality; As a reference standard for labels; For the first Semantic information of each modality Label reference benchmark The similarity function value measures the degree of matching between modal semantic information and baseline labels; based on the cross-modal semantic information parsed by the pre-trained multimodal large model, accurate alignment and dynamic calibration of semantic content of different modalities are achieved, and the semantic association characteristics of each modality are integrated to generate a label set with unified standards.

[0012] Furthermore, the specific steps in the tag linkage matching module to decompose the theme into a set of demand tags and perform multi-dimensional matching with the standardized tag set to select highly adaptable materials are as follows: extract the elements of the short video creation theme, combine the creation scenario and expression needs, expand and generate related tags, and integrate them to form a structured set of demand tags; compare the fit between the demand tag set and the standardized tag set from three dimensions: tag coreness, semantic relevance, and scene adaptability, and establish a two-way matching association; sort out the adaptability based on the multi-dimensional matching results, remove materials with low relevance, and retain materials that are highly compatible with the theme and demand tags to form a set of highly adaptable materials.

[0013] Furthermore, the tag-material-theme linkage model in the tag linkage matching module adopts a layered architecture design, specifically including an input layer, an association layer, a verification layer, and an output layer. The input layer receives standardized tag sets, demand tag sets, highly adaptable materials, and short video creation themes, completing the classification and import of multi-source information and format adaptation. The association layer contains dual association units: the theme-tag association unit establishes a bidirectional mapping relationship between the creation theme and the demand tag set and standardized tag set; the tag-material association unit binds standardized tags with corresponding highly adaptable materials, forming a linkage link between tags, materials, and themes. The verification layer performs rationality verification on the linkage link, checks the accuracy of the association between tags and materials / themes, and eliminates logically conflicting association relationships. The output layer integrates the verified linkage information and outputs candidate material combinations that meet the theme requirements.

[0014] Furthermore, in the material pre-selection pool module, the specific steps for storing materials by type and sorting them according to matching degree, and performing secondary verification are as follows: extract the type attributes of each material in the candidate material combination, divide the storage location according to the material type of visual, auditory, and text, and import each material into its corresponding storage location for classification and archiving; retrieve the matching data of each material with the short video creation theme, sort the materials stored by type according to the matching degree from high to low, and form an ordered material sequence; for the sorted material sequence, verify from three dimensions: material completeness, tag matching consistency, and theme adaptation relevance, mark the materials that fail the verification and trigger the re-matching process, and retain the materials that pass the verification.

[0015] Compared with existing technologies, this AI short video intelligent splicing system with multi-source heterogeneous materials has the following beneficial effects: I. This invention establishes a comprehensive material preprocessing workflow to fully process multi-source heterogeneous original materials, achieving unified formats for various materials, efficient removal of redundant information, calibration of key parameters, and semantic consistency verification, thus ensuring the standardization level of materials from the source. Simultaneously, in the feature extraction stage, it not only extracts the basic features of various materials but also deeply mines the semantic correspondence, temporal matching, and emotional affinity relationships between different materials, generating cross-material related features. Through a collaborative fusion algorithm, it achieves efficient integration of basic and related features, outputting a comprehensive fused feature vector. This effectively solves the problems of inconsistent material quality and one-sided feature representation in traditional technologies, providing reliable data support for subsequent label mapping and material matching, and improving the data quality of the splicing system.

[0016] Second, this invention uses a pre-trained multimodal large model to parse cross-modal semantic information and combines it with a dynamic label mapping calibration mechanism to achieve accurate alignment of cross-modal semantics and standardized label generation, ensuring the uniformity and accuracy of labels. In the material matching stage, the creative theme is decomposed into a structured set of requirement labels, and highly adaptable materials are selected through multi-dimensional label matching. The association verification is completed through a hierarchical label-material-theme linkage model. After classification, orderly sorting, and secondary verification of the material pre-selection pool, unqualified materials are re-matched, comprehensively ensuring the high adaptability of candidate materials to the creative theme. This solves the problems of insufficient label standardization, low theme matching accuracy, and lack of dual guarantee of material quality in traditional technologies, improves the material adaptation efficiency of short video creation and the coherence and consistency of the final splicing effect, and meets the diversified and high-quality creative needs.

[0017] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0019] Figure 1 A block diagram showing the modular components of an AI short video intelligent splicing system with multi-source heterogeneous materials; Figure 2 A flowchart of an AI short video intelligent splicing system for multi-source heterogeneous materials; Figure 3 This is a schematic diagram of the data transmission and processing of the heterogeneous feature extraction module in an AI short video intelligent splicing system for multi-source heterogeneous materials. Detailed Implementation

[0020] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0021] Example

[0022] The material preprocessing module collects video clips, including records of ingredient processing, dynamic displays of the store environment, and shots of dishes being served. These video clips can intuitively present the food preparation process and the store atmosphere. Images include close-ups of finished dishes, displays of ingredient freshness, and detailed images of the store's decor. Close-up images highlight the color and texture of the dishes, while decor images convey the store's style. Audio includes chef explanations, background noise from the store environment, and background music appropriate for the food scene. Explanations supplement preparation details, background noise enhances realism, and background music creates an appetizing atmosphere. A text script includes an introduction to the core highlights of the store visit. Four categories of original, heterogeneous materials, including descriptions of dish flavors and ingredient sourcing, are used in the text script to provide information support for the short video and clarify the core creative concept. The collected materials undergo format standardization processing, unifying the video's encoding format, resolution, and frame rate to ensure compatibility and adaptation in subsequent processing and avoid processing anomalies caused by format differences. Image formats and color gamuts are standardized for consistent color presentation and a more harmonious visual experience. Audio sampling rates are standardized to ensure smooth audio playback without noise interference. The encoding and text formats of the text scripts are standardized to facilitate the parsing and retrieval of text information and improve processing efficiency. The process involves: removing black frames, still frames, and invalid transition segments from videos to reduce the storage resources consumed by invalid data and make video footage more concise; removing watermarks, blank edges, and duplicate pixels from images to ensure visual cleanliness, highlight core content, and avoid redundant elements that distract attention; removing silent and noisy segments from audio to improve audio clarity and make effective sound information stand out; removing repetitive sentences from text scripts to simplify text content and avoid information redundancy affecting subsequent semantic analysis; and calibrating the color space, brightness, and contrast of the video to ensure that the video image has realistic colors, appropriate brightness, and sharp contrast, thus improving visual appeal. The process involves: calibrating image pixel density to ensure clear and detailed display across different devices; calibrating audio volume equalization to ensure consistent volume across various audio formats and prevent some audio from being too loud or too soft; calibrating text script character encoding consistency to prevent garbled text due to encoding issues and ensure accurate transmission of text information; and verifying the semantic consistency between video footage and embedded subtitles, image visual content and additional descriptions, and audio content and text scripts to ensure that the information conveyed by various materials is consistent and logically consistent, allowing for smoother integration of subsequent materials. After passing the verification, standardized heterogeneous materials are output.

[0023] The heterogeneous feature extraction module receives standardized heterogeneous materials and extracts visual features such as dish colors and shop decoration styles, auditory features such as speaking speed and background music rhythm, and textual features such as descriptive keywords and core information points. This basic feature data accurately captures the core attributes of various materials, providing the original basis for subsequent correlation analysis and feature fusion. Through correlation analysis techniques, the module analyzes the semantic, temporal, and emotional connections between different materials, extracts the semantic content of each standardized heterogeneous material, and establishes semantic correspondences between them. This ensures that food processing videos, descriptions of dish flavors, and close-up images of dishes create semantic resonance, guaranteeing coherent information delivery. For video materials with temporal attributes, the module marks time node information and matches non-temporal elements such as images and text. The timing correspondence between materials and time-series materials is determined to establish the temporal relationship between them. This ensures that video clips of food washing, preparation, and cooking are accurately aligned with corresponding step descriptions and key step images on the timeline, making the narrative logic of the short videos clearer. The emotional tendency characteristics of various standardized heterogeneous materials are identified, and the consistency of emotional tendencies among different materials is compared to clarify the emotional connections between them. This ensures that cheerful background music, appetizing food images, enthusiastic audio explanations, and positive text descriptions create emotional resonance, enhancing the appeal of the short videos. Semantic, temporal, and emotional relationship data are integrated to generate cross-material association features. These features comprehensively reflect the inherent connections between various materials, making subsequent feature fusion more targeted. A multi-source heterogeneous collaborative feature fusion algorithm is used, with the formula: ,in, This is the final output fused feature vector; These are the normalized weighting coefficients; Basic feature vectors; This method is used to generate cross-material related feature vectors. It integrates basic feature data and cross-material related features, fully combining the core attributes of single materials with the information related to multiple materials. This makes up for the limitations of single features and allows the output fused feature vector to contain the essential features of various materials and reflect the collaborative relationship between materials. This provides comprehensive and accurate feature support for cross-modal semantic parsing and finally outputs the fused feature vector.

[0024] The cross-modal label mapping module receives the fused feature vector and calls a pre-trained multimodal large model. The model's feature receiving layer splits and adapts the receiving formats of visual, auditory, and textual features, ensuring that different types of fused features can be accurately received and avoiding information loss due to format incompatibility. The cross-modal encoding layer contains multiple cascaded encoding units, deeply capturing the semantic relationships between different modal features and completing feature encoding, thus establishing a close association between visual features of dishes and textual features describing flavors and cooking sounds. The feature fusion layer optimizes and integrates the encoded multimodal features through attention weight allocation, prioritizing features highly relevant to the food exploration theme and weakening secondary information. The semantic decoding layer accurately parses the cross-modal semantic information in the fused features, outputting structured semantic representation information, transforming ambiguous feature data into clear and identifiable semantic content. A cross-modal dynamic label mapping calibration algorithm is used, with the following formula: ,in, For a single calibrated tag in a standardized tag set; The total number of modes participating in the calibration; For modal indexing; For the first The cross-modal semantic information representation value of each modality; For the first Dynamic weighting coefficients for each modality; As a reference standard for labels; For the first Semantic information of each modality Label reference benchmark The similarity function value is used to align and calibrate cross-modal semantic information, eliminate the bias in semantic expression of different modalities, and enable the tags to accurately correspond to the core content of the material, generating a standardized tag set.

[0025] The tag-linked matching module receives standardized tag sets and short video creation themes related to food exploration. It extracts the elements of the themes, combines them with food display scenarios and the expressive needs of restaurant exploration, and expands to generate related tags, integrating them into a structured demand tag set. This tag set clarifies the core direction and key points of short video creation, making material selection systematic. From three dimensions—tag coreness, semantic relevance, and scene adaptability—the module compares the fit between the demand tag set and the standardized tag set, establishing a two-way matching relationship. This multi-dimensional comparison method comprehensively considers the fit between materials and the theme, avoiding selection bias caused by single-dimensional judgment and ensuring more accurate selection results. Based on the multi-dimensional matching results, the module analyzes the adaptability, eliminating materials with low relevance to reduce interference from invalid materials in subsequent processes, improving production efficiency, and retaining materials that highly match the theme and demand tags, forming a highly adaptable material set to provide high-quality materials for short video creation. Material reserves; a linkage model for tags, materials, and themes is established. This model adopts a layered architecture design. The input layer receives standardized tag sets, demand tag sets, highly adaptable materials, and short video creation themes, ensuring that all kinds of key information are fully included and providing sufficient data for linkage analysis. The association layer contains dual association units. The theme-tag association unit establishes a two-way mapping relationship between creation themes and demand tag sets and standardized tag sets, tightly binding themes and tags. The tag-material association unit binds standardized tags with corresponding highly adaptable materials, forming a linkage link between tags, materials, and themes, making the three form an organic whole. The verification layer performs rationality verification on the linkage link, checks the accuracy of the association between tags, materials, and themes, eliminates logically conflicting associations, and avoids situations where materials do not match themes or tags and materials are misaligned. The output layer integrates the verified linkage information and outputs candidate material combinations that meet the theme requirements.

[0026] The material pre-selection pool module receives candidate material combinations, extracts the type attributes of each material, and divides the storage locations according to visual, auditory, and text material types. Each material is then imported into its corresponding storage location for categorized archiving. This categorized storage method makes the management of different types of materials more organized, allowing for quick location during subsequent retrieval, reducing search time, and improving production efficiency. The module retrieves the matching data between each material and the food exploration creation theme, sorting the stored materials by matching score from highest to lowest to form an ordered material sequence. This sorting prioritizes the materials with the highest theme relevance, ensuring the high-quality presentation of the core content of the short video. The sorted material sequence undergoes a second verification from three dimensions: material completeness, tag matching consistency, and theme adaptability. This comprehensive check ensures that the selected materials have no missing content, accurate tag matching, and a high degree of theme adaptation. Materials that fail verification are marked and a re-matching process is triggered to promptly fill gaps left by unqualified materials, preventing material issues from affecting the quality of the short video. Materials that pass verification are retained, ultimately generating an optimized material retrieval list, forming the candidate material pre-selection pool.Figure 1 As shown, the materials in this pre-selection pool have undergone multiple rounds of screening and verification, ensuring reliable quality and strong adaptability. They provide solid support for AI short video intelligent splicing, ensuring that the spliced ​​food exploration short videos have a clear theme, coherent content, and excellent quality.

[0027] Example

[0028] The preprocessing module acquires video clips, including time-lapse footage of scenic spots, tour route recordings, and dynamic displays of landmark buildings. Time-lapse footage showcases the changing day and night landscapes, cloud formations, and other dynamic beauty of the attractions. Tour route recordings present the tour path, guiding viewers through an immersive experience. Dynamic displays of landmark buildings highlight the core scenic views. Images include panoramic views, detailed views, and seasonally limited-time images. Panoramic views showcase the grandeur of the attractions, allowing viewers to intuitively grasp the overall environment. Detailed views highlight unique details, and seasonally limited-time images emphasize the scenic highlights of different seasons. Audio includes natural environmental sound effects such as flowing water and wind, audio guides, and background music. Environmental sound effects recreate the authentic atmosphere of tourist attractions, immersing viewers in the experience. Audio explanations of scenic spots convey historical, cultural, and geographical knowledge, enriching the content of short videos. Soft background music enhances emotions and improves the viewing experience. The text scripts include four types of original, heterogeneous materials: travel route introductions, historical background texts, and check-in prompts. Travel route introductions provide a logical framework for the short video narrative, historical background texts add cultural depth, and check-in prompts guide viewers to focus on key attractions. The collected materials undergo format standardization processing, unifying the video encoding format, resolution, and frame rate to ensure seamless integration of various video materials in subsequent processing, preventing playback stuttering and image distortion due to format differences. The image format and color gamut of the images are also standardized. To ensure a unified color style for images from different sources, resulting in a more harmonious visual presentation and avoiding jarring color contrasts; to standardize audio sampling rates, guaranteeing smooth playback, clear sound quality, and natural transitions between different audio formats; to unify the encoding and text format of text scripts, facilitating rapid system parsing of text information and ensuring smooth collaborative processing of text with other modal materials; to remove black frames, still frames, and invalid transition segments from videos, streamlining video content, eliminating meaningless visuals, and focusing video footage on the core scenic beauty; to remove watermarks, blank edges, and repetitive pixel blocks from images, ensuring clean and uncluttered images that highlight the main landscape and avoid redundant elements affecting the viewing experience; and to remove silent and noisy segments from audio, improving audio purity and ensuring effective sound quality. Audio information, such as natural sound effects and narration, is highlighted; repetitive sentences in the text script are removed, core information is condensed, information piling up is avoided, and text utilization efficiency is improved; the color space, brightness, and contrast of the video are calibrated to ensure that the video images reproduce the colors of the real scenery, with moderate brightness and sharp contrast, showcasing the original magnificence and beauty of the scenic spot; the pixel density of the images is calibrated to ensure that the details of the images are clearly distinguishable when displayed on different terminal devices, without blurring or distortion; the volume balance of the audio is calibrated to ensure that the volume ratio of natural sound effects, narration audio, and background music is coordinated, so that they do not interfere with each other and complement each other; the character encoding consistency of the text script is calibrated to prevent the text from being unable to be parsed properly due to encoding errors, ensuring that the text information is accurately transmitted.Verify the semantic consistency between video footage and embedded subtitles, visual content and additional descriptions in images, and audio content and text scripts. This ensures that the information conveyed by various materials corroborates each other and is logically consistent, avoiding discrepancies between visuals and text, or disconnects between audio and content. This allows for more natural integration of subsequent materials. After successful verification, standardized heterogeneous materials are output.

[0029] The heterogeneous feature extraction module receives standardized heterogeneous materials and extracts their visual features (e.g., landscape colors, architectural styles), auditory features (e.g., natural sound volume, narration clarity), and textual features (e.g., attraction names, historical keywords). This basic feature data accurately captures the core attributes of various materials, providing solid raw data support for subsequent correlation analysis and allowing the system to clearly grasp the core value of each type of material. Through correlation analysis techniques, the module analyzes the semantic, temporal, and emotional connections between different materials, extracting the semantic content of each standardized heterogeneous material and establishing semantic correspondences between them. This allows attraction images to semantically resonate with corresponding historical background text and audio narration, enabling viewers to comprehensively understand the attraction's connotation through multimodal materials. For video materials with temporal attributes, the module marks time node information and matches the time correspondence between non-temporal materials such as images and text and temporal materials to determine the temporal relationship between them. For example, it matches a panoramic sunrise image to the sunrise time node of a time-lapse video, making the material presentation more logical and coherent, and aligning with the viewer's expectations. The system analyzes and integrates various standardized heterogeneous materials to identify their emotional characteristics, compares the consistency of emotional tendencies across different materials, clarifies the emotional connections between materials, and ensures that magnificent landscape videos, soothing background music, and beautiful text descriptions maintain consistency in emotional expression, creating a unified atmosphere of tranquility, joy, or awe, thus enhancing the appeal of short videos. It integrates semantic, temporal, and emotional relational data to generate cross-material relational features. These features comprehensively reflect the intrinsic connections between various materials, breaking down the isolation of different modal materials and providing rich relational information for subsequent feature fusion. A multi-source heterogeneous collaborative feature fusion algorithm is used to collaboratively fuse basic feature data and cross-material relational features, fully integrating the core attributes of single materials with the relational information of multiple materials. This compensates for the deficiencies of a single feature dimension, ensuring that the output fused feature vector contains both the essential features of various materials and reflects the synergistic effect between materials, making the feature information more comprehensive and representative. This provides accurate and efficient feature support for cross-modal semantic parsing, ultimately outputting a fused feature vector, such as... Figure 3 As shown.

[0030] The cross-modal label mapping module receives the fused feature vector and calls a pre-trained multimodal large model. The model's feature receiving layer splits and adapts the receiving formats of visual, auditory, and textual features, ensuring that different types of fused features are accurately and completely received, avoiding feature information loss or distortion due to format adaptation issues. The cross-modal encoding layer contains multiple cascaded encoding units, deeply capturing the semantic relationships between different modal features and completing feature encoding, allowing the relationships between landscape visual features, natural sound effects features, and historical text features to be fully explored. The feature fusion layer, through attention weight allocation, processes the encoded multimodal features... The optimization and integration prioritizes core features highly relevant to the travel scenery theme, weakens the influence of secondary features, and improves feature quality. The semantic decoding layer accurately parses cross-modal semantic information in the fused features, outputting structured semantic representation information, transforming abstract feature data into clear and understandable semantic content, and providing a clear basis for tag generation. The cross-modal dynamic tag mapping calibration algorithm is used to align and calibrate cross-modal semantic information, eliminating deviations and inconsistencies in the semantic expressions of different modalities, so that the generated tags can accurately and uniformly correspond to the core content of the material, avoiding the situation where tags are disconnected from the material, and generating a standardized tag set.

[0031] The tag-linked matching module receives standardized tag sets and short video creation themes related to travel and scenery. It extracts the elements of the theme, combines them with the needs of travel display scenarios and scenery presentation, and expands to generate related tags, integrating them into a structured demand tag set. This tag set clearly defines the core direction, key points, and expressive needs of short video creation, making the material selection process clear and targeted. From three dimensions—tag coreness, semantic relevance, and scene adaptability—the module compares the fit between the demand tag set and the standardized tag set, establishing a two-way matching relationship. This multi-dimensional comparison method comprehensively and deeply considers the degree of fit between materials and the theme, avoiding the one-sidedness of single-dimensional judgment, ensuring that the selected materials not only fit the core theme but also meet the scene expression needs. Based on the multi-dimensional matching results, the module analyzes the adaptability, eliminating materials with low relevance to reduce interference from invalid materials in subsequent processes, lower production costs, improve overall efficiency, and retain materials that highly match the theme and demand tags, forming a highly adaptable material set for short video creation. Prepare high-quality, relevant material resources; establish a linkage model for tags, materials, and themes. This model adopts a layered architecture design. The input layer receives standardized tag sets, demand tag sets, highly adaptable materials, and short video creation themes, ensuring that all kinds of key information are comprehensively and accurately included in the model, providing sufficient and reliable data support for linkage analysis. The association layer contains dual association units. The theme-tag association unit establishes a two-way mapping relationship between creation themes and demand tag sets and standardized tag sets, allowing themes and tags to be deeply bound. The tag-material association unit binds standardized tags with corresponding highly adaptable materials, forming a linkage link between tags, materials, and themes, making the three closely connected and organically unified. The verification layer performs rationality verification on the linkage link, checks the accuracy of the association between tags and materials and themes, eliminates logically conflicting association relationships, and avoids problems such as materials not matching themes or tags and materials being misaligned, ensuring the logic and coherence of material combinations. The output layer integrates the verified linkage information and outputs candidate material combinations that meet the theme requirements.

[0032] The material pre-selection pool module receives candidate material combinations, extracts the type attributes of each material, and divides the storage locations according to visual, auditory, and text material types. Each material is then imported into its corresponding storage location for categorized archiving. This categorized storage makes the management of different types of materials more organized, allowing for quick location of the required material type during subsequent retrieval, thus improving material retrieval efficiency. The module also retrieves matching data between each material and the travel scenery creation theme, sorting the stored materials by matching score from highest to lowest to form an ordered material sequence. This sorting prioritizes the selection of materials with the highest theme relevance, ensuring the selection of core landscapes and key elements for the short video. Information is presented optimally to improve the overall viewing quality. For the sorted material sequence, a second verification is performed from three dimensions: material completeness, tag matching consistency, and theme adaptability and relevance. This comprehensively checks for potential issues such as missing content, misaligned tags, and thematic inconsistencies, ensuring that the selected materials are complete, accurately tagged, and highly adapted to the theme. Materials that fail verification are marked and a re-matching process is triggered to promptly supplement qualified materials, preventing material defects from affecting the overall effect of the short video. Materials that pass verification are retained, and finally, an optimized material call list is generated, forming a candidate material pre-selection pool. Figure 2 As shown, the materials in this pre-selection pool have undergone multiple rounds of rigorous screening and verification, ensuring high quality and strong adaptability. This provides a stable and high-quality material guarantee for AI short video intelligent splicing, ensuring that the spliced ​​travel scenery short videos are beautiful, information-rich, logically coherent, and emotionally unified, bringing viewers an excellent viewing experience.

[0033] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An AI short video intelligent splicing system for multi-source heterogeneous materials, characterized in that, The system comprises the following components: a material preprocessing module, a heterogeneous feature extraction module, a cross-modal label mapping module, a label linkage matching module, and a material pre-selection pool module; The material preprocessing module collects four types of raw heterogeneous materials: video clips, images, audio, and text scripts, and performs format standardization, redundant information removal, parameter calibration, and semantic verification to output standardized heterogeneous materials. The heterogeneous feature extraction module: receives standardized heterogeneous materials, extracts the basic visual, auditory, and textual features of the standardized heterogeneous materials to form basic feature data; based on the basic feature data, analyzes the semantic, temporal, and emotional correlation between different materials through association analysis technology to generate cross-material association features; and uses a multi-source heterogeneous collaborative feature fusion algorithm to collaboratively fuse the basic feature data and the cross-material association features to output a fused feature vector. The cross-modal label mapping module: receives the fused feature vector, calls the pre-trained multimodal large model, parses the cross-modal semantic information in the fused feature vector; and uses the cross-modal dynamic label mapping calibration algorithm to align and calibrate the cross-modal semantic information to generate a standardized label set. The tag linkage matching module receives a standardized tag set and an externally input short video creation theme, breaks down the theme into a demand tag set and performs multi-dimensional matching with the standardized tag set to filter out highly compatible materials; establishes a tag-material-theme linkage model, and outputs candidate material combinations after verifying the highly compatible materials. The material pre-selection pool module: receives candidate material combinations, stores them by type, sorts them according to matching degree, and performs secondary verification; For materials that fail the verification, a re-matching process is triggered, which ultimately generates an optimized list of materials to form a candidate material pre-selection pool.

2. The AI ​​short video intelligent splicing system for multi-source heterogeneous materials according to claim 1, characterized in that, The process of outputting standardized heterogeneous materials in the material preprocessing module is as follows: Video clips, images, and audio are acquired through local storage and terminal device interfaces; text scripts are acquired through database retrieval and creation platform interfaces; video clips are standardized in encoding format, resolution, and frame rate; images are standardized in image format and color gamut; audio samples are standardized in sampling rate; and text scripts are standardized in encoding format and text format. Black frames, still frames, and invalid transition segments in video clips are removed; watermarks, blank edges, and duplicate pixel blocks in images are removed; silent and noisy segments in audio are cleared; and duplicate statements in text scripts are deleted. The color space, brightness, and contrast of the video are calibrated; the pixel density of images is adjusted; audio volume is balanced; and the character encoding of text scripts is standardized. The semantic consistency between video footage and embedded subtitles, image visual content and additional descriptions, and audio content and text scripts is verified. After successful verification, standardized heterogeneous materials are output.

3. The AI ​​short video intelligent splicing system for multi-source heterogeneous materials according to claim 1, characterized in that, The heterogeneous feature extraction module uses association analysis technology to analyze the semantic, temporal, and sentiment correlations between different materials and generate cross-material association features. The process is as follows: extract the semantic content of each standardized heterogeneous material and establish semantic correspondences between heterogeneous materials; for materials with temporal attributes, mark time node information, match the time correspondence positions of non-temporal materials and temporal materials, and determine the temporal relationship between materials; identify the sentiment tendency features of each standardized heterogeneous material, compare the consistency of sentiment tendencies of different materials, and clarify the sentiment correlation between materials; integrate the correlation data of semantic, temporal, and sentiment dimensions to generate cross-material association features.

4. The AI ​​short video intelligent splicing system for multi-source heterogeneous materials according to claim 1, characterized in that, The calculation formula for the multi-source heterogeneous collaborative feature fusion algorithm in the heterogeneous feature extraction module is as follows: ,in, This is the final output fused feature vector; These are the normalized weighting coefficients; Basic feature vectors; This is a cross-material related feature vector.

5. The AI ​​short video intelligent splicing system for multi-source heterogeneous materials according to claim 1, characterized in that, The pre-trained multimodal large model in the cross-modal label mapping module specifically includes a feature receiving layer, a cross-modal encoding layer, a feature fusion layer, and a semantic decoding layer; the feature receiving layer is used to receive fused feature vectors, split and adapt the receiving formats of visual, auditory and textual modal features; A cross-modal coding layer contains multiple cascaded coding units that capture the semantic relationships between features of different modalities and complete feature encoding. The feature fusion layer integrates the encoded multimodal features through attention weight allocation; The semantic decoding layer parses the cross-modal semantic information in the fused features and outputs structured semantic representation information.

6. The AI ​​short video intelligent splicing system for multi-source heterogeneous materials according to claim 1, characterized in that, The calculation formula for the cross-modal dynamic label mapping calibration algorithm in the cross-modal label mapping module is as follows: ,in, For a single calibrated tag in a standardized tag set; The total number of modes participating in the calibration; For modal indexing; For the first The cross-modal semantic information representation value of each modality; For the first Dynamic weighting coefficients for each modality; As a reference standard for labels; For the first Semantic information of each modality Label reference benchmark The similarity function value.

7. The AI ​​short video intelligent splicing system for multi-source heterogeneous materials according to claim 1, characterized in that, The specific steps in the tag linkage matching module to decompose the theme into a set of demand tags and perform multi-dimensional matching with the standardized tag set to select highly adaptable materials are as follows: extract the elements of the short video creation theme, combine the creation scenario and expression needs, expand and generate related tags, and integrate them to form a structured set of demand tags; compare the fit between the demand tag set and the standardized tag set from three dimensions: tag coreness, semantic relevance, and scene adaptability, and establish a two-way matching association; sort out the adaptability based on the multi-dimensional matching results, remove materials with low relevance, and retain materials that are highly compatible with the theme and demand tags to form a set of highly adaptable materials.

8. The AI ​​short video intelligent splicing system for multi-source heterogeneous materials according to claim 1, characterized in that, The tag-material-theme linkage model in the tag linkage matching module adopts a layered architecture design, specifically including an input layer, an association layer, a verification layer, and an output layer. The input layer is used to receive standardized tag sets, demand tag sets, highly adaptable materials, and short video creation themes. The association layer contains dual association units: the theme-tag association unit establishes a bidirectional mapping relationship between the creation theme and the demand tag set and standardized tag set; the tag-material association unit binds standardized tags with corresponding highly adaptable materials, forming a linkage link between tags, materials, and themes. The verification layer performs rationality verification on the linkage link, checks the accuracy of the association between tags and materials / themes, and eliminates logically conflicting association relationships. The output layer integrates the verified linkage information and outputs a combination of candidate materials that meet the theme requirements.

9. The AI ​​short video intelligent splicing system for multi-source heterogeneous materials according to claim 1, characterized in that, In the material pre-selection pool module, the materials are categorized and stored by type and sorted according to matching degree. The specific steps for performing secondary verification are as follows: extract the type attributes of each material in the candidate material combination, divide the storage location according to the material type of visual, auditory, and text, and import each material into its corresponding storage location for classification and archiving; retrieve the matching data of each material with the short video creation theme, sort the materials stored by type according to the matching degree from high to low, and form an ordered material sequence; for the sorted material sequence, verify from three dimensions: material completeness, tag matching consistency, and theme adaptation relevance, mark the materials that fail the verification and trigger the re-matching process, and retain the materials that pass the verification.

Images