Intelligent video automatic synthesis method and system based on multi-material fusion

Abstract

The application relates to the technical field of digital media, and discloses an intelligent video automatic synthesis method and system based on multi-material fusion, which comprises the following steps: screening a multi-source material set from original media data, and extracting the category and tendency label of the material; constructing a material semantic pedigree according to the category label, distinguishing opposite material combination from the remaining material set, and arranging the remaining material into a preliminary sequence; combining the tendency label of the preliminary sequence to generate a target tendency path, screening the materials in the opposite material that match the path inflection point and inserting the materials into the corresponding position to obtain a harmonious material sequence; adding a time label based on the material integrity boundary to generate a time boundary material sequence; completing transition superposition according to the adjacent material head and tail marks to demarcate a safe connection interval, and generating a continuous protection audio and video stream; extracting an audio flow dynamic signal, modulating and packaging to obtain a target video, and the application aims to solve the problems of semantic connection harshness and insufficient narrative coherence of materials in synthesis.

Description

Technical Field

[0001] This invention relates to the field of digital media technology, and in particular to an intelligent video automatic synthesis method and system based on multi-material fusion. Background Technology

[0002] With the development of the digital media industry, the demand for multi-source intelligent video compositing technology in the field of video signal processing continues to increase. Typically, a tag extraction module categorizes multiple source materials, sorts them according to simple content classification rules, then stitches them together along the timeline and overlays common transition effects, ultimately packaging them into a finished video. Some automated compositing solutions employ a division-of-labor architecture, automating processes such as material recognition, segment trimming, and audio-visual encapsulation, enabling batch production of standardized videos and effectively reducing the labor costs of manual editing.

[0003] However, existing technologies, which rely solely on shallow category tags to arrange materials, cannot accurately identify the opposing semantic combinations between materials, nor can they reasonably place related materials in conjunction with the turning points of content orientation. This directly leads to insufficient logical coherence in the synthesized video content, abrupt transitions in orientation, and the overall narrative fluency and content harmony after the fusion of multi-source materials are insufficient to meet the production requirements of high-quality videos. Summary of the Invention

[0004] This invention provides an intelligent video automatic synthesis method and system based on multi-material fusion, the main purpose of which is to solve the problems of awkward semantic connection and insufficient narrative coherence in the synthesis of materials.

[0005] To achieve the above objectives, the present invention provides an intelligent video automatic synthesis method and system based on multi-source fusion, comprising: S1: Filter the multi-source material set from the original media data, and extract the category tags and trend tags from the multi-source material set; S2: Based on the category labels, construct the semantic spectrum of the materials in the multi-source material set, and based on the semantic spectrum of the materials, identify the opposing material combinations and the remaining material set among the materials in the multi-source material set, and arrange the materials in the remaining material set into a preliminary material sequence; S3: Based on the tendency tags in the preliminary material sequence, generate the target tendency path of the preliminary material sequence, select inflection point matching materials from the opposing material combination that match the inflection point position of the target tendency path, and insert the inflection point matching materials into the preliminary material sequence according to the inflection point position of the target tendency path to obtain the harmonious material sequence; S4: Based on the integrity boundary time of each material in the harmonious material sequence, add time markers to each material in the harmonious material sequence to obtain the time boundary material sequence of the harmonious material sequence; S5: Based on the end markers of the first segment and the beginning markers of the second segment of adjacent materials in the time boundary material sequence, define the safe connection interval of the adjacent materials, and perform transition superposition on the adjacent materials based on the safe connection interval to obtain the continuous protected video stream and continuous protected audio stream of the two adjacent materials; S6: Extract the audio dynamic signal of the continuous protection audio stream, and combine the audio dynamic signal to modulate and encapsulate the continuous protection video stream into a target video.

[0006] Preferably, constructing the semantic hierarchy of the materials in the multi-source material set based on the category tags includes: Decompose the semantics of the category tags to obtain the material semantic primitives of the category tags, mark the attributes of the material semantic primitives, and obtain the standard semantic primitive set of the material semantic primitives; Retrieve conflicting primitive pairs of material primitives within the standard semantic primitive set, and qualitatively characterize the material relationships within the conflicting primitive pairs to obtain the material relationship tag set of the standard semantic primitive set; The materials in the multi-source material set are instantiated into nodes to obtain the material node set of the multi-source material set. The inheritance relationship of the materials in the material relationship tag set is constructed to obtain the inheritance edge set of the material relationship tag set. Construct the collaborative relationships of materials within the material relationship tag set to obtain the collaborative edge set of the material relationship tag set; Based on the receiving edge set and the collaborative edge set, the materials in the material node set are spliced ​​together to form a material semantic spectrum.

[0007] Preferably, the step of identifying the opposing material combinations and the remaining material set among the materials in the multi-source material set based on the semantic spectrum of the materials includes: Extract the set of relation tags in the semantic spectrum of the material into a set of opposing relation edges, and construct the opposing semantic vector pairs of the set of opposing relation edges based on the node tags of the set of opposing relation edges; Based on the spectral distance metric basis of the semantic spectrum of the material, the opposing semantic distance value of the multidimensional semantic coordinates in the opposing semantic vector pair is measured; The opposing semantic distance value is injected into the opposing relation edge set to obtain the intensity-annotated opposing edge set. The initial set of opposing material pairs in the intensity-annotated opposing edge set that exceed the global conflict tolerance benchmark of the material semantic spectrum is then selected. Based on the collaborative constraint relationship of the semantic spectrum of the materials, the initial set of opposing material pairs is verified to obtain the combination of opposing materials; The opposing material combinations are removed from the multi-source material set to obtain a smooth material set. Based on the semantic connectivity of the smooth material set, isolated material nodes in the smooth material set are recycled to obtain the remaining material set.

[0008] Preferably, generating the target tendency path of the preliminary material sequence based on the tendency tags in the preliminary material sequence includes: The tendency primitive vectors of the tendency labels in the preliminary material sequence are labeled to obtain the tendency primitive vector sequence; Based on the material arrangement order in the preliminary material sequence, the tendency contrast intensity of adjacent tendency primitive vectors in the tendency primitive vector sequence is identified, and a tendency contrast intensity value sequence of the tendency primitive vector sequence is obtained. Mark the inverted nodes of the trend contrast intensity value sequence to obtain the inverted node index set of the trend primitive vector sequence; Based on the aforementioned tendency primitive vector sequence, the tendency inertia of the reverse nodes in the reverse node index set is smoothed to obtain a smooth tendency vector sequence of the reverse node index set. The path of the smooth tendency vector is then concatenated to obtain the target tendency path.

[0009] Preferably, the step of filtering inflection point matching materials from the combination of opposing materials that match the inflection point position of the target tendency path includes: Extract the inflection point position of the target tendency path to obtain the tendency reversal vector at the inflection point position; Based on the tendency tag sequence of the opposing material combination, the oscillation time sequence of the opposing material combination is identified, and the material oscillation time sequence spectrum of the opposing material combination is obtained; By matching the tendency reversal vector at the inflection point position with the material oscillation time series spectrum of the opposing material combination, a preliminary set of opposing materials is obtained for the opposing material combination; Align the trend reversal time code of the initial screening opposing material set with the sequence timestamp of the inflection point position to obtain the candidate inflection point material set of the initial screening opposing material set; Based on the inertial trajectory spectrum of the candidate inflection point material set and the inflection point inertial trajectory spectrum of the target inflection path, the inertial coherence of the candidate inflection point material set is verified to obtain the inflection point matching material.

[0010] Preferably, inserting the inflection point matching material into the preliminary material sequence according to the inflection point position of the target tendency path to obtain the harmonious material sequence includes: Based on the sequence timestamp marked by the tendency reversal vector in the target tendency path, the preliminary material sequence is segmented to obtain the sequence segmentation scheme of the preliminary material sequence; Based on the last frame of the preceding subsequence in the sequence segmentation scheme, extract the neighboring frames of the inflection point blank time slot in the sequence segmentation scheme to obtain the connection reference frame of the preceding subsequence. The transition bridge between the connecting reference frame and the starting frame of the inflection point matching material is obtained by transitioning and fusing the transition reference frame and the starting frame of the inflection point matching material. Based on the connecting transition bridge and the inflection point blank time slot, the inflection point matching material is embedded into the preliminary material sequence to obtain the seamless reconstructed transient sequence of the preliminary material sequence; Based on the tendency label of the seamless recombination transient sequence and the target tendency path, the duration of the seamless recombination transient sequence is adjusted to obtain the harmonious material sequence of the preliminary material sequence.

[0011] Preferably, the step of adding time stamps to each material in the harmonious material sequence based on the integrity boundary time of each material in the harmonious material sequence to obtain the time boundary material sequence of the harmonious material sequence includes: The multidimensional semantics of the harmonious material sequence are analyzed to obtain the boundary marker set of each material in the harmonious material sequence; The boundary marker set is conflict-resolved and merged to obtain the fusion boundary time of each material in the harmonious material sequence; Based on the content persistence of the fusion boundary moment, the fusion boundary moment is verified and corrected to obtain the candidate boundary moment of each material in the harmonious material sequence; Add time stamps to the candidate boundary moments and associate them with the original materials in the original media data to generate the time boundary material sequence of the harmonious material sequence.

[0012] Preferably, defining the safe connection interval between adjacent materials based on the end markers of the preceding segment and the beginning markers of the following segment in the time boundary material sequence includes: The initial connection window of the adjacent materials is obtained by dividing the front end marker and the rear beginning marker of adjacent materials in the time boundary material sequence; Based on the end marker of the front segment and the beginning marker of the rear segment, the semantic fading point and semantic starting point in the adjacent material are located to obtain the semantic transition anchor point of the adjacent material; Based on the semantic transition anchor points of the materials, the boundaries of the initial connection window are migrated to obtain the safe connection interval of the adjacent materials.

[0013] Preferably, the step of transitioning and overlaying adjacent materials based on the secure connection interval to obtain a continuous protected video stream and a continuous protected audio stream for the two adjacent materials includes: Within the safe transition interval, the last frame sequence of the first segment of the adjacent material and the first frame sequence of the second segment of the adjacent material are pixel-weightedly synthesized frame by frame, and the screen switching rate is dynamically adjusted according to the severity of the difference in screen content between adjacent frames in the adjacent material to generate a transition video segment. Within the secure connection interval, the non-overlapping video segments of the preceding material, the transition video segment, and the non-overlapping video segments of the following material are spliced ​​together according to the timeline of the time boundary material sequence to obtain a continuous protection video stream. Extract the short-term energy envelope of the end of the preceding audio segment and the short-term energy envelope of the beginning of the following audio segment from the adjacent materials. Within the safe connection interval, perform linear energy attenuation on the preceding audio segment and simultaneously perform an equal amount of energy gain on the following audio segment to generate a continuous protection audio stream.

[0014] To address the aforementioned problems, this invention also provides an intelligent video automatic synthesis system based on multi-source fusion, comprising: Material Tag Extraction Module: Filters a multi-source material set from the original media data and extracts category tags and trend tags from the multi-source material set; Material type identification module: Based on the category tags, construct the material semantic spectrum of the materials in the multi-source material set, and based on the material semantic spectrum, identify the opposing material combinations and the remaining material set among the materials in the multi-source material set, and arrange the materials in the remaining material set into a preliminary material sequence; Material positioning and insertion module: Based on the tendency tags in the preliminary material sequence, generate the target tendency path of the preliminary material sequence, select inflection point matching materials from the opposing material combination that match the inflection point position of the target tendency path, and insert the inflection point matching materials into the preliminary material sequence according to the inflection point position of the target tendency path to obtain the harmonious material sequence; Time boundary marker module: Based on the integrity boundary time of each material in the harmonious material sequence, add time markers to each material in the harmonious material sequence to obtain the time boundary material sequence of the harmonious material sequence; Safe transition overlay module: Based on the end markers of the first segment and the beginning markers of the second segment of adjacent segments in the time boundary material sequence, the safe connection interval of the adjacent segments is defined, and the adjacent segments are overlaid based on the safe connection interval to obtain two segments of the adjacent segments of continuous protected video stream and continuous protected audio stream; Target video synthesis module: Extracts the audio dynamic signal of the continuous protection audio stream, and combines the audio dynamic signal to modulate and encapsulate the continuous protection video stream into a target video.

[0015] Beneficial effects This solution uses methods such as decomposing material category tags to build a content association network and calculating the degree of content difference to accurately select material combinations with strong content opposition. At the same time, it sorts out the remaining material sequences that can be arranged in sequence, realizing the classification management of materials from multiple sources and the advance sorting of narrative logic. It solves the problems of existing technologies that rely solely on simple classification tags for sorting, cannot identify opposing content materials, and are prone to logical breaks in content connection. It has the benefits of improving the rationality of material arrangement and strengthening the overall narrative coherence of the video.

[0016] This solution analyzes the content tendencies of the materials to generate an overall trend path, matches corresponding transitional materials, and smoothly inserts them into the corresponding positions. This not only makes full use of the content value of various materials, but also avoids the content tendency transitions being too abrupt, making the final video's expression rhythm more natural, the narrative thread clearer, and significantly improving the overall content harmony and viewing smoothness.

[0017] This solution uses multiple perspectives to analyze the content and define effective boundaries, and adds unified time markers to accurately locate the start and end positions of the core content of each segment of material. It achieves precise extraction of the effective content of the material and unified management of the time sequence, solving the problem that transition overlays in existing technologies can easily intrude into the core content of the material and damage its integrity. It has the benefits of ensuring the integrity of the material content and improving the security of transitions.

[0018] This solution adjusts the transition range by modifying content transition nodes, dynamically adjusts the screen switching speed, and synchronously processes audio volume changes to ensure a smooth and continuous transition between the screen and audio. It also combines audio rhythm to calibrate the video playback rhythm, ensuring that the audio and video are synchronized throughout. The final synthesized video screen transitions are natural, the audio transitions are smooth, and the overall viewing experience is smooth and comfortable. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating an intelligent video automatic synthesis method based on multi-material fusion, provided in an embodiment of the present invention. Figure 2 A functional block diagram of an intelligent video automatic synthesis system based on multi-material fusion provided in an embodiment of the present invention; The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0021] This application provides a method and system for automatic intelligent video synthesis based on multi-source fusion. The executing entity of this method and system includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method and system for automatic intelligent video synthesis based on multi-source fusion can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0022] Reference Figure 1 The diagram shown is a flowchart illustrating an intelligent video automatic synthesis method based on multi-source fusion according to an embodiment of the present invention. In this embodiment, the intelligent video automatic synthesis method based on multi-source fusion includes: S1: Filter the multi-source material set from the original media data, and extract the category tags and trend tags from the multi-source material set; Specifically, raw media data refers to the data files corresponding to various unstructured and unedited raw media signals stored in digital storage media. It covers video signal segments, audio signal segments, and still image signal files with different encoding standards and serves as the basic data source for material selection and subsequent video compositing.

[0023] A multi-source material collection is a set of reusable materials retained from the original media data after usability screening. The materials in the collection have different sources and content themes, and all meet the format compatibility requirements and content quality requirements for video synthesis. They are the direct processing objects for subsequent tag extraction and content semantic analysis.

[0024] Category tags are content classification identifiers generated for a single piece of material. They are used to mark the theme category and content type to which the core content carried by the material belongs, providing a classification dimension to support the subsequent construction of the semantic spectrum of the material.

[0025] Tendency tags are content tendency identifiers generated for a single piece of material. They are used to mark the emotional orientation, narrative emphasis, and expressive stance conveyed by the material content, providing a basis for judging the tendency dimension for subsequent target tendency path generation and inflection point material matching.

[0026] Furthermore, all original media data stored in the digital storage medium is read, and format compatibility checks, content integrity checks, and duration compliance checks are performed on each piece of original media data. Format compatibility verification specifically confirms that the encoding format of the original media data can be read and processed normally in subsequent video compositing stages.

[0027] Content integrity detection specifically involves confirming that the video frames of the original media data are undamaged and unmissing, and that the audio signals are uninterrupted and undistorted.

[0028] The duration compliance assessment specifically involves confirming that the total duration of the original media data meets the minimum usable requirements for video synthesis.

[0029] The original media data that simultaneously meets the above three requirements are selected and retained from the overall data, and then a multi-source material set is formed. Subsequently, the content of each material in the multi-source material set is analyzed one by one.

[0030] Read the continuous video frame content and corresponding audio signal content in the material, match the corresponding category label for each material according to the core subject and the theme category, and match the corresponding tendency label for each material according to the narrative focus and emotional expression direction.

[0031] S2: Based on the category labels, construct the semantic spectrum of the materials in the multi-source material set, and based on the semantic spectrum of the materials, identify the opposing material combinations and the remaining material set among the materials in the multi-source material set, and arrange the materials in the remaining material set into a preliminary material sequence; In this embodiment, constructing the semantic hierarchy of materials in the multi-source material set based on the category labels includes: Decompose the semantics of the category tags to obtain the material semantic primitives of the category tags, mark the attributes of the material semantic primitives, and obtain the standard semantic primitive set of the material semantic primitives; Retrieve conflicting primitive pairs of material primitives within the standard semantic primitive set, and qualitatively characterize the material relationships within the conflicting primitive pairs to obtain the material relationship tag set of the standard semantic primitive set; The materials in the multi-source material set are instantiated into nodes to obtain the material node set of the multi-source material set. The inheritance relationship of the materials in the material relationship tag set is constructed to obtain the inheritance edge set of the material relationship tag set. Construct the collaborative relationships of materials within the material relationship tag set to obtain the collaborative edge set of the material relationship tag set; Based on the receiving edge set and the collaborative edge set, the materials in the material node set are spliced ​​together to form a material semantic spectrum.

[0032] Specifically, the semantics of category tags are the content classification meanings carried by the category labeling information in the material metadata, and they are the basic parsing objects for decomposing the content attributes of the material.

[0033] Material semantic primitives are the smallest set of semantic units obtained by decomposing the semantics of category labels. They are the basic building blocks that constitute the complete classification meaning of materials.

[0034] The attributes of semantic primitives are the content dimension features corresponding to each semantic primitive, which are the identification information used to standardize the representation of semantic primitives.

[0035] The standard semantic primitive set is a collection of all semantic primitives of materials after attribute tagging is completed. It serves as a unified benchmark for subsequent retrieval of material associations.

[0036] Conflicting primitive pairs are paired units consisting of two semantic primitives in the standard semantic primitive set that have semantic opposition or mutual exclusion. They are the basic judgment units for identifying the opposition relationship of materials.

[0037] The specific process of qualitatively classifying material relationships is as follows: For each pair of conflicting primitives retrieved, the video content orientation, expressive tendency, and narrative adaptation attributes corresponding to the two semantic primitives are compared one by one to determine the nature of their association. If the core semantics of the two semantic primitives are opposite and their content expressions are mutually exclusive, a direct sequential connection in a continuous video narrative would create logical gaps and visual conflicts, thus classifying them as opposing relationships. If the two semantic primitives have a clear logical sequence, and the video content corresponding to the former primitive is the narrative premise, progressive link, or causal groundwork for the content corresponding to the latter primitive, then classifying them as successive relationships. If the core semantics of the two semantic primitives are in the same direction and their content dimensions are complementary, and their combined presentation in the video can jointly support the expression effect of the same theme, then classifying them as synergistic relationships. Standardized identifiers corresponding to the relationship type are matched for each pair of primitives, and all relationship identifiers that have been qualitatively labeled are summarized to finally form a set of material relationship tags.

[0038] The material relationship tag set is a set of relationship identifiers formed after qualitatively labeling the material relationships of all conflicting primitive pairs. It is a summary set that records the relational properties between semantic primitives.

[0039] Node instantiation is the process of converting actual footage into independent nodes in a semantic network; it's the step of mapping entity footage into network structural elements. Specifically, the instantiation process involves traversing each independent video clip in the multi-source footage set, assigning a unique node identifier to each clip, and simultaneously attaching the clip's category label, tendency label, and associated semantic primitive information to the corresponding node. This transforms each entity footage into an independent node unit with complete attribute information in the semantic network. All nodes are then aggregated to form a footage node set.

[0040] The material node set is a collection of all network nodes corresponding to the materials after node instantiation is completed. It is the basic node unit that constitutes the semantic spectrum of the materials.

[0041] The sequential relationship is a logical progression and continuation of content between different material nodes, and it is a type of relationship that describes the rationality of the playback order of materials.

[0042] The set of connecting edges is a collection of all connecting edges between material nodes that have a connecting relationship. It is the vertical connecting unit for constructing the semantic spectrum of the material.

[0043] Collaborative relationships are relationships between different material nodes that are complementary in content, echoing in theme, and can be presented side by side. This type of relationship describes the adaptability of material content.

[0044] The collaborative edge set is a collection of all connecting edges between material nodes that have collaborative relationships. It is a horizontal connecting unit for constructing the semantic spectrum of materials.

[0045] The semantic spectrum of materials is a network semantic structure composed of material node sets, receiving edge sets, and cooperative edge sets. It uses materials as nodes and semantic receiving and cooperative relationships between materials as edges. It is a structured carrier used to represent the semantic association and arrangement logic between materials.

[0046] Furthermore, the category labeling information of all materials in the multi-source material set is read, and the semantics of each labeling information is decomposed layer by layer to the smallest indivisible semantic unit. All the smallest semantic units are summarized to form a material semantic primitive group. Then, the corresponding dimension attribute label is matched for each semantic primitive, the expression standard of all semantic primitives is unified, and all semantic primitives with completed attribute labeling are summarized to obtain the standard semantic primitive set.

[0047] Then, iterate through all semantic primitives in the standard semantic primitive set, compare the meanings of semantic primitives in pairs, select primitives with mutually exclusive semantic content to form conflicting primitive pairs, determine the relationship type of each pair of conflicting primitives, add corresponding qualitative relationship identifiers to each pair of primitives, and summarize all qualitative relationship identifiers to obtain the material relationship tag set.

[0048] Next, each independent material in the multi-source material set is mapped to an independent network node, and the semantic primitive information of the corresponding material is matched to each node. All nodes are then aggregated to form a material node set.

[0049] Then, based on the association rules in the material relationship tag set, we determine whether there is a logical progression and continuation relationship between different material nodes. We add connecting edges to two nodes with a succession relationship, and summarize all succession-type connecting edges to obtain the succession edge set.

[0050] Based on the association rules in the material relationship tag set, we determine whether there is a relationship between different material nodes that is complementary in content, echoes the theme, and can be presented side by side. We add connecting edges to two nodes that have a collaborative relationship, and summarize all collaborative connecting edges to obtain a collaborative edge set.

[0051] Ultimately, the material node set is used as the basic node unit of the network, and the receiving edge set and the cooperative edge set are used as different types of connection paths. All nodes are spliced ​​together through the corresponding connection edges according to the relationship between the nodes to form a material semantic spectrum with a complete semantic association structure.

[0052] In this embodiment, identifying the opposing material combinations and the remaining material set among the materials in the multi-source material set based on the material semantic spectrum includes: Extract the set of relation tags in the semantic spectrum of the material into a set of opposing relation edges, and construct the opposing semantic vector pairs of the set of opposing relation edges based on the node tags of the set of opposing relation edges; Based on the spectral distance metric basis of the semantic spectrum of the material, the opposing semantic distance value of the multidimensional semantic coordinates in the opposing semantic vector pair is measured; The opposing semantic distance value is injected into the opposing relation edge set to obtain the intensity-annotated opposing edge set. The initial set of opposing material pairs in the intensity-annotated opposing edge set that exceed the global conflict tolerance benchmark of the material semantic spectrum is then selected. Based on the collaborative constraint relationship of the semantic spectrum of the materials, the initial set of opposing material pairs is verified to obtain the combination of opposing materials; The opposing material combinations are removed from the multi-source material set to obtain a smooth material set. Based on the semantic connectivity of the smooth material set, isolated material nodes in the smooth material set are recycled to obtain the remaining material set.

[0053] Specifically, the relation tag set is a collection of semantic relation attribute identifiers marked on each connection edge of the semantic spectrum of the material, used to distinguish the association types between different material nodes.

[0054] The set of opposing relationships is a collection of all connecting edges labeled as opposing relationships in the semantic spectrum of the materials. The materials connected at both ends of the corresponding edge have opposing attributes at the semantic level.

[0055] Node tags are semantic attribute identifiers attached to each material node in the material semantic hierarchy, used to represent the content category and core semantic features of the corresponding material.

[0056] An opposition semantic vector pair is a set of two sets of semantic representation vectors corresponding to the directions obtained by semantic transformation of the labels of the nodes at both ends of the same opposition relationship edge. It is used to quantify the semantic opposition relationship between the two materials.

[0057] The spectral distance metric is a benchmark scale for calculating semantic distance within the semantic spectrum of the material, used to uniformly measure the degree of semantic difference between different semantic vectors.

[0058] Multidimensional semantic coordinates are the position coordinates of opposite semantic vector pairs in the multidimensional semantic space of the material's semantic spectrum, used to characterize the attribute distribution state of the material from multiple semantic dimensions.

[0059] The opposing semantic distance value is a quantitative value of the difference between opposing semantic vector pairs in multidimensional semantic coordinates, calculated based on the spectral distance metric. The larger the value, the stronger the semantic opposition between the two materials.

[0060] The intensity-annotated opposition edge set is a set of opposition edges where each opposition edge is annotated with a corresponding semantic distance value, used to intuitively reflect the opposition intensity level of each set of materials.

[0061] The global conflict tolerance benchmark is a threshold for determining the intensity of conflict within the semantic spectrum of the material. It is used to determine whether the degree of conflict between material pairs reaches the standard that requires separate handling.

[0062] The global conflict tolerance benchmark is calculated by taking the arithmetic mean of all opposing semantic distances corresponding to all opposing relationships within the semantic spectrum of the source material, and then adding one standard deviation. The result is set as the global conflict tolerance benchmark. Specifically, this benchmark can be adjusted up or down based on the narrative harmony requirements of the target video to adapt to different video compositing needs.

[0063] The initial set of opposing material pairs is a collection of all material pairs whose opposing semantic distance values ​​in the intensity-annotated opposing edge set exceed the global conflict tolerance benchmark. This is the high-opposition material combination obtained from the initial screening.

[0064] Collaborative constraint relationships are the associated constraints such as content collaboration and scene adaptation that exist between different material nodes in the semantic spectrum of materials. They are used to verify the actual opposition validity of material pairs.

[0065] A set of opposing material pairs is a collection of material pairs that have been verified to have a valid opposing semantic relationship after the collaborative constraint relationship has been checked. It is a group of material pairs that needs to be placed in a separate location.

[0066] A multi-source material set is a collection of all media materials from different sources that are selected from the original media data and are to be used in video synthesis.

[0067] A smooth material set is a collection of materials remaining after removing opposing material combinations from a multi-source material set. The overall semantic tendency of the materials in the set is smooth and there is no strong semantic opposition or conflict.

[0068] Semantic connectivity is the degree to which the nodes of each material in a smooth material set are interconnected through semantic connection edges and collaborative edges, and is used to determine whether a material can be integrated into the overall semantic narrative.

[0069] An isolated material node is an independent material node in a smooth material set that does not have semantic connection or collaborative relationship with other material nodes and cannot be directly integrated into the semantic context of the existing material sequence.

[0070] The remaining material set is the material set obtained by recycling and adjusting the smooth material set through isolated nodes. The materials in the set have a coherent semantic relationship and can be directly used to arrange them to form a preliminary material sequence.

[0071] Furthermore, the relational labels of all connecting edges are traversed from the constructed semantic spectrum of the materials. All connecting edges marked as opposing types are extracted and summarized one by one to form a set of opposing relational edges. At the same time, the node labels of the nodes at both ends of each opposing relational edge are read, and the semantic labels of each pair of opposing nodes are converted into semantic representation vectors in the corresponding directions. The two pairs are then paired to form opposing semantic vector pairs.

[0072] Each pair of opposing semantic vectors is mapped to a multidimensional semantic coordinate space. Combined with the preset spectral distance metric of the material semantic spectrum, the positional differences under the multidimensional semantic coordinates are quantified and calculated to obtain the opposing semantic distance value corresponding to each pair of opposing materials.

[0073] The calculated opposing semantic distance values ​​are labeled one by one onto each opposing relationship edge to form a set of opposing edge intensity labels with opposing intensity indicators. At the same time, the global conflict tolerance benchmark of the semantic spectrum of the material is retrieved, and the opposing semantic distance values ​​of each edge in the set of opposing edge intensity labels are compared one by one. Material pairs with values ​​exceeding the global conflict tolerance benchmark are selected and summarized to form a preliminary set of opposing material pairs.

[0074] Retrieve the collaborative constraint relationships in the semantic spectrum of the materials, and verify each pair of materials in the initial set of opposing materials one by one. Material pairs that have collaborative relationships but do not have substantial opposing effects are excluded, and the remaining effective opposing material pairs are combined to form opposing material combinations.

[0075] From the original multi-source material set, all materials included in the opposing material combination are removed to obtain the initial smooth material set. Then, the semantic connectivity of all material nodes in the smooth material set is checked to identify isolated material nodes that have no semantic connection with other materials. Isolated material nodes that meet the semantic continuity conditions are reintegrated into the semantic context of the smooth material set to finally obtain the remaining material set.

[0076] S3: Based on the tendency tags in the preliminary material sequence, generate the target tendency path of the preliminary material sequence, select inflection point matching materials from the opposing material combination that match the inflection point position of the target tendency path, and insert the inflection point matching materials into the preliminary material sequence according to the inflection point position of the target tendency path to obtain the harmonious material sequence; In this embodiment, generating the target tendency path of the preliminary material sequence based on the tendency tags in the preliminary material sequence includes: The tendency primitive vectors of the tendency labels in the preliminary material sequence are labeled to obtain the tendency primitive vector sequence; Based on the material arrangement order in the preliminary material sequence, the tendency contrast intensity of adjacent tendency primitive vectors in the tendency primitive vector sequence is identified, and a tendency contrast intensity value sequence of the tendency primitive vector sequence is obtained. Mark the inverted nodes of the trend contrast intensity value sequence to obtain the inverted node index set of the trend primitive vector sequence; Based on the aforementioned tendency primitive vector sequence, the tendency inertia of the reverse nodes in the reverse node index set is smoothed to obtain a smooth tendency vector sequence of the reverse node index set. The path of the smooth tendency vector is then concatenated to obtain the target tendency path.

[0077] Specifically, the tendency primitive vector sequence is an ordered set of vectors formed by breaking down the tendency label of each material in the initial material sequence into the smallest tendency semantic units and arranging them in the order of the material's arrangement. It is used to reflect the distribution of the content tendency of the entire material sequence along the playback sequence.

[0078] The order of material arrangement is the sequential playback order of each video material in the initial material sequence, determined by semantic connection. It serves as the baseline order for conducting trend-based temporal analysis.

[0079] Adjacent tendency primitive vectors are two sets of tendency primitive vectors that are immediately adjacent in the playback sequence, corresponding to the smallest tendency representation unit of two consecutive video clips.

[0080] The intensity of the contrast in tendencies is a quantitative value of the degree of difference between two adjacent sets of tendency primitive vectors in terms of emotional orientation, narrative stance, and expressive emphasis. The larger the value, the greater the magnitude of the shift in the content tendency of the two connecting materials.

[0081] The trend contrast intensity value sequence is an ordered set of values ​​formed by arranging the trend contrast intensity values ​​corresponding to all adjacent vectors in the trend primitive vector sequence in the order of the material sequence.

[0082] A reversal node is a point in the sequence of contrast intensity values ​​where the contrast intensity value exceeds the threshold for determining a turning point. This point corresponds to a point in the sequence of materials where the content trend changes significantly, and is a point in the sequence where the content trend undergoes a substantial change.

[0083] The inverted node index set is a collection of the position indices of all inverted nodes in the trend primitive vector sequence, used to accurately locate the specific temporal position of all trend reversals in the material sequence.

[0084] Smoothing involves referencing the trend of the tendency of adjacent materials before and after the reversal node, gradually narrowing the difference in tendency on both sides of the reversal node, adjusting the original abrupt tendency change to a gradual transition, preserving the narrative logic of the overall tendency shift, eliminating the abruptness brought by the tendency jump, and making the tendency change of the entire material sequence present a coherent and smooth extension.

[0085] Tendency inertia is the characteristic of the continuity of the content's tendency in the playback sequence. It is manifested in the fact that the tendency within a single piece of material remains stable and the tendency transition between adjacent pieces of material has the attribute of a continuous trend.

[0086] A smoothed tendency vector sequence is an ordered set of all tendency primitive vectors in the entire sequence after the tendency primitive vectors at the inverted node positions have been transitioned and adjusted. The transitions are smooth and the transitions are natural.

[0087] The target tendency path is a continuous tendency change trajectory formed by sequentially connecting smooth tendency vector sequences in time sequence. It is used to characterize the overall change trend of the content tendency that the final harmonious material sequence should possess.

[0088] Furthermore, when calibrating the tendency primitive vector sequence of the tendency labels of the preliminary material sequence, the tendency labels of each video material in the preliminary material sequence are read one by one, each tendency label is decomposed into the smallest indivisible tendency semantic unit, and the corresponding tendency representation dimension value is matched for each smallest tendency semantic unit.

[0089] Then, strictly following the material arrangement order of the initial material sequence, all the tendency representation units corresponding to the materials are arranged sequentially to form an ordered vector set.

[0090] Based on the order of materials in the initial material sequence, the tendency primitive vectors of each group are extracted sequentially according to the order of playback. The differences between the two groups of vectors in terms of emotional orientation, narrative emphasis, and expressive stance are compared one by one to obtain the quantitative value of the tendency difference corresponding to each group of adjacent vectors.

[0091] Then, all the quantitative values ​​are arranged in the original order of the material to form an ordered sequence of tendency contrast intensity values.

[0092] Next, iterate through all the values ​​in the trend contrast intensity value sequence, calculate the arithmetic mean of the values ​​in the entire sequence and add one standard deviation of the values ​​to obtain the turning point judgment threshold. Compare each value with the turning point judgment threshold one by one, and mark the corresponding position where the value exceeds the turning point judgment threshold as the reversal node.

[0093] Simultaneously, record the position number of each inverted node in the biased primitive vector sequence, and summarize all position numbers to form an inverted node index set.

[0094] Finally, based on the overall trend of the tendency primitive vector sequence, the tendency representation at each reversal node position is processed to mitigate the tendency jump amplitude at the reversal node, so that the tendency change before and after the reversal node presents a continuous and smooth transition state. After the adjustment, a smooth tendency vector sequence with natural overall transition is obtained.

[0095] Then, all vectors in the smoothing tendency vector sequence are connected sequentially in chronological order to form a continuous content tendency change trajectory, i.e., the target tendency path.

[0096] In this embodiment, the step of filtering inflection point matching materials from the opposing material combination that match the inflection point position of the target tendency path includes: Extract the inflection point position of the target tendency path to obtain the tendency reversal vector at the inflection point position; Based on the tendency tag sequence of the opposing material combination, the oscillation time sequence of the opposing material combination is identified, and the material oscillation time sequence spectrum of the opposing material combination is obtained; By matching the tendency reversal vector at the inflection point position with the material oscillation time series spectrum of the opposing material combination, a preliminary set of opposing materials is obtained for the opposing material combination; Align the trend reversal time code of the initial screening opposing material set with the sequence timestamp of the inflection point position to obtain the candidate inflection point material set of the initial screening opposing material set; Based on the inertial trajectory spectrum of the candidate inflection point material set and the inflection point inertial trajectory spectrum of the target inflection path, the inertial coherence of the candidate inflection point material set is verified to obtain the inflection point matching material.

[0097] Specifically, the inflection point is the temporal point where the video content tendency takes a substantial turn on the target tendency path. It corresponds to the connection point where the narrative tendency of the preceding and following materials shows a clear turning point, and it is the benchmark positioning node for the insertion of opposing materials.

[0098] The tendency reversal vector is a representation of the direction and magnitude of the content tendency shift at the inflection point. It corresponds to the change in the emotional orientation and narrative emphasis of the video before and after that point, and can intuitively reflect the turning attribute of the inflection point.

[0099] A tendency tag sequence is an ordered set of tags for each video clip within a combination of opposing materials, arranged according to the playback sequence of the material's content. It is used to reflect the change in the content tendency of a single opposing material as the playback duration increases.

[0100] Oscillation timing refers to the distribution of time points in which the content tends to change repeatedly within opposing video materials. It corresponds to the sequence of time points in which positive and negative tendencies alternate during the playback of the material itself, and can reflect the rhythm of the tendency changes within the material.

[0101] The material oscillation time series spectrum is a time series distribution map formed by summarizing and organizing the oscillation time series of all materials in the opposing material combination according to a unified time dimension. It can intuitively present the time nodes and change rhythms of the tendency reversal within each opposing material.

[0102] The initial screening of the opposing material set is a collection of video materials selected from the opposing material combinations after preliminary matching of the tendency reversal vector and the material oscillation time spectrum. These materials are those whose tendency turning features and inflection point turning features are likely to match.

[0103] The tendency reversal time code is the time code corresponding to the specific moment when the tendency reversal occurs within each piece of material in the initial screening set of opposing materials. It accurately corresponds to the specific frame position within the playback duration of the material and can accurately locate the turning point within the material.

[0104] A sequence timestamp is a time marker on the overall timeline of the initial material sequence corresponding to the inflection point of the target tendency path. It is used to locate the specific position of the inflection point in the complete video playback sequence.

[0105] The candidate inflection point material set is a collection of video materials that, after time alignment verification, tend to match the timing of the reversal time with the inflection point position. It is a group of alternative materials that can be used to insert inflection point positions.

[0106] The tendency inertia trajectory spectrum is a trajectory map of the continuous change of the content tendency of the candidate inflection point material before and after the reversal node, reflecting the gradual transition state and continuity characteristics of the material tendency before and after the reversal.

[0107] The inflection point inertial trajectory spectrum is a trajectory map of the continued change in the tendency of the target tendency path before and after the inflection point, reflecting the gradual transition trend and inertial characteristics of the overall video content tendency of the material sequence before and after the turning point.

[0108] Inertial coherence is the degree of continuity and adaptation between the tendency change trajectory of the candidate inflection point material and the inflection point trajectory of the target tendency path. It is used to judge whether the overall video tendency transition is natural and smooth after the material is inserted.

[0109] Inflection point matching material is opposing video material that, after undergoing inertial coherence verification, can be smoothly connected to the inflection point position of the target tendency path, and can be accurately inserted into the corresponding inflection point position of the initial material sequence.

[0110] Furthermore, the entire trajectory of the generated target tendency path is traversed, and the locations where the tendency change rate exceeds the normal transition interval are identified segment by segment. These locations are marked as inflection points, and the change attribute of the tendency at the inflection point from the original direction to the new direction is recorded. The direction and magnitude of the change are integrated into a tendency reversal vector.

[0111] The tendency tags of all video materials in the opposing material combination are retrieved and arranged sequentially according to the playback duration of each material to form a tendency tag sequence for each material. The time nodes where the tendency alternates between positive and negative are identified segment by segment along the time axis of the material itself. All time nodes are summarized to form the oscillation time sequence of the opposing material combination. Then, the oscillation time sequences of all opposing materials are regularized and arranged according to a unified time scale to generate the material oscillation time sequence spectrum.

[0112] The turning direction and turning amplitude attributes of the tendency reversal vector corresponding to the inflection point position are compared one by one with the oscillation characteristics of each material in the material oscillation time series spectrum. Opposite materials whose oscillation turning direction is consistent with the direction of the tendency reversal vector and whose turning amplitude range is compatible are selected. The materials that meet the conditions are compiled to form a preliminary set of opposite materials.

[0113] Extract the specific frame positions where the tendency reverses within each piece of material in the initial screening set of opposing materials, convert the frame positions into the tendency reversal time codes of the corresponding material's own duration dimension, and at the same time retrieve the sequence timestamps corresponding to the inflection point positions on the overall timeline of the initial material sequence.

[0114] Alignment verification is performed between the trend reversal time code and the sequence timestamp of each material in the time dimension. Materials whose reversal time can accurately correspond to the inflection point time sequence position are selected and summarized to form a candidate inflection point material set.

[0115] Extract the trajectory of the trend continuation change of each material in the candidate inflection point material set before and after the trend reversal node, and organize it into a trend inertial trajectory spectrum. At the same time, extract the trajectory of the trend continuation change of the target trend path before and after the inflection point position, and organize it into an inflection point inertial trajectory spectrum.

[0116] The changing trends and transition rates of the two trajectory spectra are compared segment by segment to verify the degree of inertial continuity after their connection. The material with the best inertial continuity is selected as the final inflection point matching material.

[0117] In this embodiment, inserting the inflection point matching material into the preliminary material sequence according to the inflection point position of the target tendency path to obtain a harmonious material sequence includes: Based on the sequence timestamp marked by the tendency reversal vector in the target tendency path, the preliminary material sequence is segmented to obtain the sequence segmentation scheme of the preliminary material sequence; Based on the last frame of the preceding subsequence in the sequence segmentation scheme, extract the neighboring frames of the inflection point blank time slot in the sequence segmentation scheme to obtain the connection reference frame of the preceding subsequence. The transition bridge between the connecting reference frame and the starting frame of the inflection point matching material is obtained by transitioning and fusing the transition reference frame and the starting frame of the inflection point matching material. Based on the connecting transition bridge and the inflection point blank time slot, the inflection point matching material is embedded into the preliminary material sequence to obtain the seamless reconstructed transient sequence of the preliminary material sequence; Based on the tendency label of the seamless recombination transient sequence and the target tendency path, the duration of the seamless recombination transient sequence is adjusted to obtain the harmonious material sequence of the preliminary material sequence.

[0118] Specifically, the tendency reversal vector is the tendency direction change representation information extracted from the inflection point of the target tendency path. It corresponds to a specific node on the time axis and is used to mark the node attributes and change direction of the content tendency in the material sequence, carrying the tendency change magnitude and turning logic at the inflection point.

[0119] A sequence timestamp is a time marker that marks the position of each key node in a sequence of materials on the overall playback timeline. It uses video frame numbers or playback duration values ​​as carriers to accurately locate the temporal position of turning and connecting nodes in the sequence of materials, providing a time reference for material segmentation and insertion.

[0120] The sequence segmentation scheme is a segmentation planning result obtained by segmenting the initial material sequence based on the inflection point time sequence position. It includes the time sequence range, connection position and blank insertion time slot information of each sub-sequence, providing a segmentation execution basis for material insertion.

[0121] The preceding subsequence is a collection of material fragments located before the inflection point insertion position in the sequence segmentation scheme. It is located in the first half of the overall material sequence timeline, and its end content is the reference content for connecting the images with the inserted material.

[0122] The last frame of the preceding subsequence is the last video frame in the playback sequence of the preceding subsequence. It is a single static pixel image and serves as the final frame of the preceding subsequence content, as well as the starting reference frame for connecting with subsequently inserted materials.

[0123] The inflection point blank time slot is a time interval reserved in the sequence segmentation scheme for placing inflection point matching material. It is located between the preceding subsequence and the following subsequence on the time axis, and its timing length is adapted to the playback duration of the inflection point matching material.

[0124] The frames near the end of the preceding subsequence of the inflection point blank time slot are used to extract the reference frame information for the transition processing.

[0125] The transition reference frame is a standard reference frame extracted from the position of the neighboring frame for carrying out transition processing. It carries the pixel features and content attributes of the image at the end of the preceding sub-sequence and is the reference input image for image transition fusion.

[0126] Transition blending is a process that mixes the pixels of two consecutive video clips. Within a set transition time interval, the pixel ratio of the preceding and following images is adjusted frame by frame, so that the preceding image gradually fades and the following image gradually appears, generating a continuous and gradual transition image to ensure the smoothness of the connection between the two clips.

[0127] Inflection point matching material is video material selected from the combination of opposing materials that matches the inflection point position of the target tendency path. It is used to place at the tendency turning point position of the material sequence to connect the tendency change logic of the preceding and following segments.

[0128] The starting frame is the first video frame in the playback sequence of the inflection point matching material. It is a single static pixel image and serves as the starting frame of the inflection point matching material content, acting as a connecting frame to the preceding sub-sequence.

[0129] The transition bridge is a transitional frame segment obtained through transition fusion processing, which is a transitional frame segment between the preceding sub-sequence and the inflection point matching material. It contains multiple frames of continuously gradually changing pixel images, arranged along the time axis between the two segments of material, eliminating the abruptness of the image jump and achieving a smooth and natural transition between the two segments of material images.

[0130] Embedding is the operation of placing the inflection point matching material along with the connecting transition part into the corresponding time slot position of the initial material sequence, completing the position alignment along the time axis, so that the inserted material and the original material sequence form a unified whole in terms of timing and image.

[0131] The seamless reconstruction transient sequence is a temporary material sequence obtained by embedding the inflection point matching material into the initial material sequence. It completes the position splicing of the material and the initial connection of the picture. The duration needs to be further adjusted to adapt to the requirements of the tendency path.

[0132] Tendency tags are content tendency identifiers generated for a single piece of material. They correspond to the time span of the material and are used to mark the emotional orientation, narrative emphasis, and expressive stance conveyed by the material content, providing a basis for judging the tendency dimension when adjusting the duration.

[0133] The adjustment process involves comparing and seamlessly reconstructing the transient sequence segment by segment to identify deviations in the trend of the target trend path. Without compromising the integrity of the core content of the material, the duration of each segment is corrected by trimming redundant frames at the beginning and end of the material and adjusting the frame playback rate. This ensures that the trend change trajectory of the entire sequence perfectly matches the target trend path, guaranteeing a coherent and suitable overall narrative rhythm.

[0134] The duration of a seamless reconstruction transient sequence is the playback time of video footage from the start frame to the end frame, quantified in seconds or frames. It is a core parameter for measuring the time span of footage and directly affects the overall rhythm and rate of change of the footage sequence.

[0135] The harmonious sequence of materials is the final sequence of materials obtained after inserting the inflection point material and adjusting the duration. It has overall semantic coherence, natural transition of tendencies, complete narrative logic, and smooth visual effect.

[0136] Furthermore, firstly, locate the sequence timestamps corresponding to all trend reversal vectors in the target trend path, and use each timestamp as a segmentation point to divide the continuous preliminary material sequence into multiple consecutive subsequences along the time axis. At the same time, reserve a blank time slot of corresponding length at each segmentation point to form a complete sequence segmentation scheme.

[0137] Then, the preceding subsequence corresponding to each segmentation point in the sequence segmentation scheme is read, the last frame of the preceding subsequence is extracted, and the boundary frame position of the blank time slot at the segmentation point is located on the side of the preceding subsequence. The preceding subsequence image corresponding to this position is used as the connection reference frame.

[0138] Next, the starting frame of the connecting reference frame and the corresponding inflection point matching material are taken. Based on the pixel content of the two frames, the proportion of the two frames in a single frame is adjusted frame by frame in the order of time progression to generate a continuous gradient transition frame segment as a connecting transition bridge.

[0139] Then, the transition bridge is placed between the end of the preceding subsequence and the beginning of the inflection point matching material. The inflection point matching material, together with the transition structures at both ends, is then placed into the inflection point blank time slot reserved in the sequence segmentation scheme, so that the inserted content is completely aligned with the original preceding and following subsequences on the time axis, splicing together to form a complete seamless reconstructed transient sequence.

[0140] Finally, the tendency tags of each segment in the seamless reconstruction transient sequence are read segment by segment. The actual tendency change trend of the sequence is compared with the target tendency path point by point. For segments with deviations in the tendency change rate, the playback speed of the material is adjusted or the effective segment range of the material is extracted to change the playback duration of the corresponding segment until the tendency change trend of the entire sequence completely matches the target tendency path, and finally a harmonious material sequence is obtained.

[0141] S4: Based on the integrity boundary time of each material in the harmonious material sequence, add time markers to each material in the harmonious material sequence to obtain the time boundary material sequence of the harmonious material sequence; In this embodiment, the step of adding time markers to each material in the harmonious material sequence based on the integrity boundary time of each material in the harmonious material sequence to obtain the time boundary material sequence of the harmonious material sequence includes: The multidimensional semantics of the harmonious material sequence are analyzed to obtain the boundary marker set of each material in the harmonious material sequence; The boundary marker set is conflict-resolved and merged to obtain the fusion boundary time of each material in the harmonious material sequence; Based on the content persistence of the fusion boundary moment, the fusion boundary moment is verified and corrected to obtain the candidate boundary moment of each material in the harmonious material sequence; Add time stamps to the candidate boundary moments and associate them with the original materials in the original media data to generate the time boundary material sequence of the harmonious material sequence.

[0142] Specifically, the multidimensional semantics of a harmonious material sequence refers to the semantic information carried by each segment of material within the harmonious material sequence in multiple dimensions, including content theme, narrative logic, visual scene, and audio rhythm. Based on the video frames and audio signals of the material, it can reflect the start and end node characteristics of the material content from different dimensions.

[0143] The boundary marker set is a set of identifiers that identify the complete start and end nodes of the content of a single piece of material from the multidimensional semantics. Each piece of material corresponds to a set of boundary markers, including the start node identifier and the end node identifier, which correspond to the specific video frame positions where the core content of the material begins and ends, respectively.

[0144] Conflict resolution and fusion is a process of coordinating and unifying the positions of boundary markers identified in different semantic dimensions of the same material. It is applied to the frame bit markers within the boundary marker set to eliminate the temporal deviation of boundary markers in different dimensions.

[0145] The fusion boundary moment is the unified start and end time of a single segment of material after conflict resolution and fusion. It is carried by video frame number or playback duration value and corresponds to the start and end time points of the core content of the material. It is the basic time parameter that represents the complete boundary of the material content.

[0146] Content persistence refers to the coverage of core content frames and the proportion of redundant frames within the content interval defined by the fusion boundary moment. It is used to measure whether the currently defined boundary moment can fully carry the core information of the material and whether there is any invalid or redundant content.

[0147] Candidate boundary moments are the start and end moments of the material that meet the content integrity requirements after content continuity verification and correction. They are the direct reference moments for adding time stamps later and correspond to the start and end frames of the final retained content of the material.

[0148] A timestamp is a standardized timestamp attached to a candidate boundary moment. It is generated using a unified time encoding rule and can accurately locate the start and end positions of the corresponding material on the overall video timeline, corresponding one-to-one with the video frames of the material.

[0149] Raw media data refers to the data files corresponding to various unstructured and unedited original media signals stored in digital storage media. It covers video and audio signal segments with different encoding standards and is the original data source for all materials.

[0150] The original material is the original media file of each segment of the corresponding harmonious material sequence in the original media data. It retains the complete original frame picture and audio signal without any cropping or modification.

[0151] A time boundary material sequence is an ordered collection of materials, each with a precise start and end time identifier after time markers have been added and associated with the original material. It is stored in a digital storage medium and serves as the direct input for subsequent transition and overlay processing.

[0152] Furthermore, each video clip in the harmonious material sequence stored in the digital storage medium is read segment by segment. The starting and ending frames of the core narrative content of the material are identified sequentially from the content theme dimension, the starting and ending frames of the scene switching of the material are identified from the scene dimension, and the corresponding frame positions where the rhythm of the accompaniment of the material changes significantly from the audio rhythm dimension. The starting and ending frame positions identified from each dimension are used as the boundary markers of the corresponding dimension.

[0153] Summarize the boundary markers of all dimensions of the same piece of material to form the boundary marker set corresponding to that material, and then extract the boundary marker sets of all materials in sequence.

[0154] For each segment of footage, the frame timing of the boundary markers corresponding to different dimensions is compared one by one. If there are differences in the timing of the starting boundary marker frames of different dimensions, the frame with the latest timing among all starting markers is selected as the unified starting boundary reference. If there are differences in the timing of the ending boundary marker frames of different dimensions, the frame with the earliest timing among all ending markers is selected as the unified ending boundary reference. This eliminates the timing conflict of boundary markers of different dimensions. The determined starting and ending reference frames are converted into duration values ​​under a unified time axis to obtain the fusion boundary time of the footage. The fusion boundary time calculation of all footage is completed in sequence.

[0155] For the fusion boundary moments of a single clip, the corresponding clip frame interval is defined. Each frame is checked to see if the interval completely contains all the corresponding frames of the clip's core content. At the same time, it is checked whether there are consecutive non-core redundant frames within the interval. If core content frames fall outside the defined interval, the boundary frame position on the corresponding side is expanded outward along the time sequence. The core content coverage is rechecked after each adjustment until all core content frames are included in the interval. If there are consecutive non-core redundant frames within the interval, the boundary frame position on the corresponding side is shrunk inward along the time sequence. While retaining all core content frames, consecutive redundant frames are removed. The start and end times after the adjustment are the candidate boundary moments of the clip. The candidate boundary moments of all clips are determined in this way.

[0156] Standardized timestamps are added to the start and end times of the candidate boundary timestamps of each clip. The timestamps use a unified time encoding rule and are consistent with the time scale of the overall video timeline. Then, the start and end times with time stamps are bound and associated with the corresponding original clip files in the original media data stored in the digital storage medium, so that the time stamps can be directly mapped to the corresponding frame positions of the original clips.

[0157] Finally, following the original order of the harmonious material sequence, all materials associated with the original materials and their completed time markers are arranged sequentially to generate a time boundary material sequence and stored in a digital storage medium.

[0158] S5: Based on the end markers of the first segment and the beginning markers of the second segment of adjacent materials in the time boundary material sequence, define the safe connection interval of the adjacent materials, and perform transition superposition on the adjacent materials based on the safe connection interval to obtain the continuous protected video stream and continuous protected audio stream of the two adjacent materials; In this embodiment, defining the safe connection interval between adjacent materials based on the end markers of the preceding segment and the beginning markers of the following segment in the time boundary material sequence includes: The initial connection window of the adjacent materials is obtained by dividing the front end marker and the rear beginning marker of adjacent materials in the time boundary material sequence; Based on the end marker of the front segment and the beginning marker of the rear segment, the semantic fading point and semantic starting point in the adjacent material are located to obtain the semantic transition anchor point of the adjacent material; Based on the semantic transition anchor points of the materials, the boundaries of the initial connection window are migrated to obtain the safe connection interval of the adjacent materials.

[0159] Specifically, adjacent clips are two independent video clips that are immediately adjacent in the playback sequence within a time boundary clip sequence. They have a direct need for content connection and are the target objects for transition overlay processing.

[0160] The front end marker is a standardized timestamp attached to the boundary moment of the previous segment in adjacent footage. It corresponds to the specific video frame position where the core content of the previous segment ends, and can accurately locate the ending node of the preceding footage on the overall video timeline.

[0161] The beginning marker of the latter segment is a standardized timestamp attached to the boundary of the beginning of the next segment of adjacent footage. It corresponds to the specific video frame position where the core content of the next segment begins, and can accurately locate the starting node of the subsequent footage on the overall video timeline.

[0162] The initial transition window is an initial time interval defined based on the end marker of the first segment and the beginning marker of the second segment of adjacent clips. It covers the overlap duration of the end of the first segment and the beginning of the second segment, and serves as the basic reference interval for subsequent adjustments to the transition interval.

[0163] The semantic decay point is the specific video frame position where the core narrative content is completed and the information density of the image begins to decrease continuously within the range of the end of the preceding material, corresponding to the time node where the influence of the preceding material content gradually weakens.

[0164] The semantic starting point is the specific video frame position where the core narrative content begins to appear and the information density of the image begins to increase within the initial range of the subsequent material, corresponding to the temporal node where the influence of the subsequent material content gradually increases.

[0165] The semantic transition anchor point is a combination of temporal reference points determined by the semantic fading point and the semantic starting point. It is used to mark the reasonable start and end nodes of the transition between two pieces of content, and to provide a precise semantic reference benchmark for adjusting the boundary of the connection interval.

[0166] The boundaries of the initial stitching window are the start and end times of the time interval corresponding to the initial stitching window, which correspond to the positions of the initially defined start and end frames of the stitching process, respectively.

[0167] The safe transition interval is a time interval obtained by adjusting the boundaries based on the semantic transition anchor point of the material. It covers the complete duration range of the natural transition between two pieces of material content, which can ensure that the entire transition and overlay process is within the interval of smooth semantic transition of content, and avoid destroying the integrity of the core content of the material.

[0168] Furthermore, the time boundary material sequence stored in the digital storage medium is read, and two adjacent material segments are identified sequentially along the timeline of the video playback. The end marker of the first segment and the beginning marker of the second segment are extracted respectively.

[0169] The number of extended frames is determined based on the duration requirement of the target transition effect and the frame rate of the source material. Using the frame positions corresponding to the two markers as a reference, the corresponding number of video frames are extended into the inner side of each of the two source materials to form an initial time range that covers the overlapping duration of the two source materials. This time range is then determined as the initial connection window for the corresponding adjacent source materials.

[0170] Traverse the consecutive video frames at the end of the previous material along the timeline, and confirm the presentation status and information carrying capacity of the core content in the frame one by one. When the core narrative content of multiple consecutive frames has been completely concluded and the effective information carrying capacity shows a continuous downward trend, mark the video frame position corresponding to the starting point of the trend as the semantic decay point.

[0171] Traverse the consecutive video frames at the beginning of the subsequent material along the time axis, and confirm the presentation status and information carrying capacity of the core content in the frame one by one. When the core narrative content of multiple consecutive frames begins to appear and the effective information carrying capacity shows a continuous upward trend, mark the video frame position corresponding to the starting point of the trend as the semantic starting point.

[0172] Pair the semantic fading points and semantic starting points corresponding to the same set of materials to form the semantic transition anchor points of the set of materials.

[0173] Using the semantic transition anchor point of each group of adjacent materials as the adjustment benchmark, the front boundary of the initial connection window is moved backward to the frame position corresponding to the semantic fading point, and the back boundary of the initial connection window is moved forward to the frame position corresponding to the semantic start point, thus forming the adjusted time interval.

[0174] After verifying the coverage of the adjusted time interval, confirming that the start frame of the interval is not earlier than the semantic fade-out point and the end frame is not later than the semantic start point, and that the interval does not intrude into the core content segment of the two clips, the time interval is determined as the safe connection interval for the group of adjacent clips.

[0175] In this embodiment, the step of transitioning and overlaying adjacent materials based on the secure connection interval to obtain a continuous protected video stream and a continuous protected audio stream for the two adjacent materials includes: Within the safe transition interval, the last frame sequence of the first segment of the adjacent material and the first frame sequence of the second segment of the adjacent material are pixel-weightedly synthesized frame by frame, and the screen switching rate is dynamically adjusted according to the severity of the difference in screen content between adjacent frames in the adjacent material to generate a transition video segment. Within the secure connection interval, the non-overlapping video segments of the preceding material, the transition video segment, and the non-overlapping video segments of the following material are spliced ​​together according to the timeline of the time boundary material sequence to obtain a continuous protection video stream. Extract the short-term energy envelope of the end of the preceding audio segment and the short-term energy envelope of the beginning of the following audio segment from the adjacent materials. Within the safe connection interval, perform linear energy attenuation on the preceding audio segment and simultaneously perform an equal amount of energy gain on the following audio segment to generate a continuous protection audio stream.

[0176] Specifically, the preceding material is a single video clip that is at the beginning of the playback sequence, and the final image and audio are the preceding input content that has been overlaid with a transition.

[0177] The final frame sequence is an ordered collection of multiple consecutive video frames that fall within the safe transition zone at the end of the preceding footage, carrying all pixel information from the final stage of the preceding footage.

[0178] The latter part of the material is a single video clip that is located later in the playback sequence. Its initial visuals and audio are the subsequent input content that has been overlaid with transitions.

[0179] The first frame sequence is an ordered collection of multiple consecutive video frames that fall within the safe connection range at the beginning of the subsequent footage, carrying all pixel information of the initial stage of the subsequent footage.

[0180] Pixel-weighted composition is a processing method that assigns different proportions of pixel weights to corresponding frames of two segments of footage at the same time position, and then adds the pixel values ​​of the two frames according to the weights to generate a single frame transition image.

[0181] The degree of difference in image content between adjacent frames is the magnitude of pixel content change between two adjacent transition frames during the transition process, used to reflect the smoothness of the scene switch. It is quantified by structural similarity, with lower values ​​indicating more severe differences. First, no less than 50 typical adjacent material transition frame pairs are extracted from a multi-source material set. The structural similarity between frames is calculated for each pair and the numerical distribution is statistically analyzed. The lower quartile of the sample is taken as the initial threshold. The two are negatively mapped: when the structural similarity is lower than the preset threshold, the single-frame weight step size is reduced and the switching rate is slowed down; when it is higher than the threshold, the step size is increased and the rate is accelerated. The total duration is constrained to not exceed the safe transition range.

[0182] The frame transition rate is the rate of change in the proportion of the preceding footage decreasing and the proportion of the following footage increasing within a unit of time. It is used to control the speed and rhythm of the transition process.

[0183] Transition video segments are sequences of multiple frames generated after pixel-weighted compositing within a safe transition interval. They are used to connect the non-overlapping parts of two segments of footage and eliminate the abrupt changes in the visuals.

[0184] A time boundary sequence is an ordered collection of clips, each with a precise start and end time marker. All clips' time markers are mapped to the same overall time scale.

[0185] The timeline is a linear time scale used to uniformly mark the temporal positions of all video footage and transition segments, using video frame numbers or playback duration as the basic units of measurement.

[0186] Non-overlapping video segments are the parts of a single clip that do not fall within the safe transition zone and do not participate in transition overlay processing, thus fully preserving the original image content and playback sequence of the clip.

[0187] The first-to-last splicing is an operation that connects the non-overlapping parts of the first segment of footage, the transition video segment, and the non-overlapping parts of the second segment of footage in sequence according to the timeline, forming a continuous and uninterrupted sequence of images.

[0188] Continuous protection video stream is a complete video signal sequence with continuous and seamless content generated after transition splicing, preserving the core content and smooth transition effect of the two segments.

[0189] The end of the preceding audio segment is the last audio segment within the safe transition zone of the audio signal attached to the preceding material, carrying all the audio information of the final stage of the preceding material.

[0190] The short-term energy envelope is a smooth curve formed by connecting the energy peaks of each frame after calculating the short-term energy of an audio signal frame by frame. It is used to reflect the change trend of the loudness of the audio signal over time.

[0191] The beginning of the second audio segment is the initial audio segment within the safe connection range of the audio signal attached to the second audio segment, carrying all the audio information of the initial stage of the second audio segment.

[0192] Linear energy decay is a processing method that reduces the energy amplitude of the preceding audio signal frame by frame along the time progression direction at a fixed ratio, so that the loudness of the preceding audio decreases uniformly.

[0193] Equal energy gain is a processing method that increases the energy amplitude of the audio signal in the subsequent segment frame by frame along the time progression, in a proportion that is exactly the same as the attenuation amplitude of the preceding audio segment, so that the loudness of the subsequent audio segment increases uniformly.

[0194] The continuous protection audio stream is a complete audio signal sequence with continuous and abrupt loudness changes generated after the audio energy transition processing is completed, and it maintains strict timing synchronization with the continuous protection video stream.

[0195] Furthermore, within the time range of the safe connection interval, the video frame transition compositing process is first carried out, extracting the images of the last frame sequence of the preceding material and the first frame sequence of the following material at the corresponding time position frame by frame.

[0196] For all pixels in each corresponding frame, assign corresponding weights to the preceding material and the remaining weights to the following material. The weight percentage gradually transitions from 100% for the preceding material and 0% for the following material over time to 0% for the preceding material and 100% for the following material. Multiply the brightness and color values ​​of the two pixels by their corresponding weights and add them together to obtain the corresponding pixel values ​​of the transition frame at that time position.

[0197] The weight allocation defaults to a linear change rule: as time progresses, the weight of the first part of the material decreases uniformly from 100% to 0, while the weight of the second part of the material increases uniformly from 0 to 100%. At the same time, a non-linear mode with slow-in and slow-out can be configured, which controls the weight rate through a cosine curve, making the transitions at the beginning and end smooth and the rate in the middle faster, thus weakening the abruptness of the boundary.

[0198] The image composition of all overlapping time periods is completed frame by frame. During the composition process, the difference in pixel content between two adjacent transition frames is continuously compared. When the difference is too large, the weight change difference between the preceding and following materials in a single frame is reduced, and the image switching rate is slowed down. When the difference is too small, the weight change difference between the preceding and following materials in a single frame is increased, and the image switching rate is accelerated, and finally a complete transition video segment is generated.

[0199] The three video segments are analyzed along the timeline. All frames from the first segment that do not enter the safe transition zone are captured as the first non-overlapping video segment, and all frames from the second segment that do not enter the safe transition zone are captured as the second non-overlapping video segment. Following the playback order from front to back, the last frame of the first non-overlapping video segment is aligned and spliced ​​with the first frame of the transition video segment. Then, the last frame of the transition video segment is aligned and spliced ​​with the first frame of the second non-overlapping video segment. The three segments are seamless and non-overlapping in time, forming a complete continuous protection video stream.

[0200] After the video processing is completed, audio transition processing is carried out simultaneously. The end segment of the audio attached to the first segment of the first segment and the beginning segment of the audio attached to the second segment of the second segment are extracted respectively. The short-time energy value of the two audio segments is calculated frame by frame, and the energy peaks of each frame are connected to form the corresponding short-time energy envelope.

[0201] Within the duration of the safe connection interval, the signal energy amplitude of the first audio segment is reduced frame by frame with a uniform step size from the start time to the end time of the interval, so that the loudness of the first audio segment drops uniformly from its original value to zero. Simultaneously, the signal energy amplitude of the second audio segment is increased frame by frame with the same step size, so that the loudness of the second audio segment rises uniformly from zero to its original value. The energy changes of the two audio segments remain complementary throughout the entire process, and finally they are merged to generate a continuous protection audio stream with no sudden changes in loudness and perfect synchronization with the video stream.

[0202] S6: Extract the audio dynamic signal of the continuous protection audio stream, and combine the audio dynamic signal to modulate and encapsulate the continuous protection video stream into a target video.

[0203] Specifically, the audio dynamic signal is the dynamic characteristic information such as the fluctuation of audio loudness and rhythm nodes that are collected segment by segment along the time axis from the continuous protected audio stream. It is used as a reference for calibrating the playback rhythm of the video stream.

[0204] Modulation and encapsulation is a process that adjusts the frame timing matching relationship of a video stream based on the rhythmic characteristics of the audio dynamic signal and integrates the video and audio signals into a single playable media file according to a standard media encoding format.

[0205] The target video is the final media file generated after audio and video synchronization modulation and format encapsulation. It has a coherent narrative logic and smooth audio-visual synchronization effect. It can be directly used for playback and distribution. It is the final processing product stored in digital storage media.

[0206] Furthermore, the continuous protected audio stream that has completed the transition and overlay processing is read, and the amplitude change information of the audio signal is collected segment by segment along a unified playback time axis at fixed frame intervals to extract the audio dynamic signal corresponding to each time node. Then, the continuous protection video stream under the same timing reference is retrieved, and the timing arrangement of each frame in the video stream is adjusted segment by segment with the fluctuation nodes of the audio dynamic signal as a reference, so that the switching nodes of the picture and the dynamic change nodes of the audio are aligned with each other, ensuring that the audio and video playback rhythm is completely synchronized. After completing the audio-visual timing matching calibration, the synchronized and calibrated continuous protection video stream and continuous protection audio stream are integrated according to the standard media encoding format. The video image data and audio signal data are mapped to the encapsulation structure of the same media file according to the corresponding timing, and finally a target video that can be played independently is generated. The target video is then written to the specified storage path of the digital storage medium for saving.

[0207] like Figure 2 The diagram shown is a functional block diagram of an intelligent video automatic synthesis system based on multi-material fusion provided in an embodiment of the present invention.

[0208] The intelligent video automatic compositing system 100 based on multi-material fusion described in this invention can be installed in an electronic device. Depending on the functions implemented, the intelligent video automatic compositing system 100 may include a material tag extraction module 101, a material type identification module 102, a material positioning and insertion module 103, a time boundary marking module 104, a safe transition overlay module 105, and a target video compositing module 106. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, stored in the memory of the electronic device.

[0209] In this embodiment, the functions of each module / unit are as follows: Material tag extraction module 101: Filters a multi-source material set from the original media data and extracts category tags and trend tags from the multi-source material set; Material type identification module 102: Based on the category label, construct the material semantic spectrum of the materials in the multi-source material set, and based on the material semantic spectrum, identify the opposing material combinations and the remaining material set among the materials in the multi-source material set, and arrange the materials in the remaining material set into a preliminary material sequence; Material positioning and insertion module 103: Based on the tendency tags in the preliminary material sequence, it generates the target tendency path of the preliminary material sequence, selects inflection point matching materials from the opposing material combination that match the inflection point position of the target tendency path, and inserts the inflection point matching materials into the preliminary material sequence according to the inflection point position of the target tendency path to obtain the harmonious material sequence; Time boundary marker module 104: Based on the integrity boundary time of each material in the harmonious material sequence, add time markers to each material in the harmonious material sequence to obtain the time boundary material sequence of the harmonious material sequence; Safe transition overlay module 105: Based on the end markers of the front segment and the beginning markers of the rear segment of adjacent segments in the time boundary material sequence, it defines the safe connection interval of the adjacent segments, and performs transition overlay on the adjacent segments based on the safe connection interval to obtain two segments of the adjacent segments of continuous protected video stream and continuous protected audio stream; Target video synthesis module 106: Extracts the audio dynamic signal of the continuous protection audio stream, and combines the audio dynamic signal to modulate and encapsulate the continuous protection video stream into a target video.

[0210] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0211] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0212] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0213] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0214] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0215] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for automatic intelligent video synthesis based on multi-source fusion, characterized in that, The method includes: S1: Filter the multi-source material set from the original media data, and extract the category tags and trend tags from the multi-source material set; S2: Based on the category labels, construct the semantic spectrum of the materials in the multi-source material set, and based on the semantic spectrum of the materials, identify the opposing material combinations and the remaining material set among the materials in the multi-source material set, and arrange the materials in the remaining material set into a preliminary material sequence; S3: Based on the tendency tags in the preliminary material sequence, generate the target tendency path of the preliminary material sequence, select inflection point matching materials from the opposing material combination that match the inflection point position of the target tendency path, and insert the inflection point matching materials into the preliminary material sequence according to the inflection point position of the target tendency path to obtain the harmonious material sequence; S4: Based on the integrity boundary time of each material in the harmonious material sequence, add time markers to each material in the harmonious material sequence to obtain the time boundary material sequence of the harmonious material sequence; S5: Based on the end markers of the first segment and the beginning markers of the second segment of adjacent materials in the time boundary material sequence, define the safe connection interval of the adjacent materials, and perform transition superposition on the adjacent materials based on the safe connection interval to obtain the continuous protected video stream and continuous protected audio stream of the two adjacent materials; S6: Extract the audio dynamic signal of the continuous protection audio stream, and combine the audio dynamic signal to modulate and encapsulate the continuous protection video stream into a target video.

2. The intelligent video automatic synthesis method based on multi-material fusion as described in claim 1, characterized in that, The step of constructing a semantic hierarchy of materials in the multi-source material set based on the category labels includes: Decompose the semantics of the category tags to obtain the material semantic primitives of the category tags, mark the attributes of the material semantic primitives, and obtain the standard semantic primitive set of the material semantic primitives; Retrieve conflicting primitive pairs of material primitives within the standard semantic primitive set, and qualitatively characterize the material relationships within the conflicting primitive pairs to obtain the material relationship tag set of the standard semantic primitive set; The materials in the multi-source material set are instantiated into nodes to obtain the material node set of the multi-source material set. The inheritance relationship of the materials in the material relationship tag set is constructed to obtain the inheritance edge set of the material relationship tag set. Construct the collaborative relationships of materials within the material relationship tag set to obtain the collaborative edge set of the material relationship tag set; Based on the receiving edge set and the collaborative edge set, the materials in the material node set are spliced ​​together to form a material semantic spectrum.

3. The intelligent video automatic synthesis method based on multi-material fusion as described in claim 1, characterized in that, The step of identifying opposing material combinations and remaining material sets among materials in the multi-source material set based on the semantic spectrum of the materials includes: Extract the set of relation tags in the semantic spectrum of the material into a set of opposing relation edges, and construct the opposing semantic vector pairs of the set of opposing relation edges based on the node tags of the set of opposing relation edges; Based on the spectral distance metric basis of the semantic spectrum of the material, the opposing semantic distance value of the multidimensional semantic coordinates in the opposing semantic vector pair is measured; The opposing semantic distance value is injected into the opposing relation edge set to obtain the intensity-annotated opposing edge set. The initial set of opposing material pairs in the intensity-annotated opposing edge set that exceeds the global conflict tolerance benchmark of the material semantic spectrum is then selected. Based on the collaborative constraint relationship of the semantic spectrum of the materials, the initial set of opposing material pairs is verified to obtain the combination of opposing materials; The opposing material combinations are removed from the multi-source material set to obtain a smooth material set. Based on the semantic connectivity of the smooth material set, isolated material nodes in the smooth material set are recycled to obtain the remaining material set.

4. The intelligent video automatic synthesis method based on multi-material fusion as described in claim 1, characterized in that, The step of generating the target trend path of the preliminary material sequence based on the trend tags in the preliminary material sequence includes: The tendency primitive vectors of the tendency labels in the preliminary material sequence are labeled to obtain the tendency primitive vector sequence; Based on the material arrangement order in the preliminary material sequence, the tendency contrast intensity of adjacent tendency primitive vectors in the tendency primitive vector sequence is identified, and a tendency contrast intensity value sequence of the tendency primitive vector sequence is obtained. Mark the inverted nodes of the tendency contrast intensity value sequence to obtain the inverted node index set of the tendency primitive vector sequence; Based on the aforementioned tendency primitive vector sequence, the tendency inertia of the reverse nodes in the reverse node index set is smoothed to obtain a smooth tendency vector sequence of the reverse node index set. The path of the smooth tendency vector is then concatenated to obtain the target tendency path.

5. The intelligent video automatic synthesis method based on multi-material fusion as described in claim 1, characterized in that, The step of filtering inflection point matching materials from the combination of opposing materials that match the inflection point position of the target tendency path includes: Extract the inflection point position of the target tendency path to obtain the tendency reversal vector at the inflection point position; Based on the tendency tag sequence of the opposing material combination, the oscillation time sequence of the opposing material combination is identified, and the material oscillation time sequence spectrum of the opposing material combination is obtained; By matching the tendency reversal vector at the inflection point position with the material oscillation time series spectrum of the opposing material combination, a preliminary set of opposing materials for the opposing material combination is obtained; Align the trend reversal time code of the initial screening opposing material set with the sequence timestamp of the inflection point position to obtain the candidate inflection point material set of the initial screening opposing material set; Based on the inertial trajectory spectrum of the candidate inflection point material set and the inflection point inertial trajectory spectrum of the target inflection path, the inertial coherence of the candidate inflection point material set is verified to obtain the inflection point matching material.

6. The intelligent video automatic synthesis method based on multi-material fusion as described in claim 1, characterized in that, The step of inserting the inflection point matching material into the preliminary material sequence according to the inflection point position of the target tendency path to obtain the harmonious material sequence includes: Based on the sequence timestamp marked by the tendency reversal vector in the target tendency path, the preliminary material sequence is segmented to obtain the sequence segmentation scheme of the preliminary material sequence; Based on the last frame of the preceding subsequence in the sequence segmentation scheme, extract the neighboring frames of the inflection point blank time slot in the sequence segmentation scheme to obtain the connection reference frame of the preceding subsequence. The transition bridge between the connecting reference frame and the starting frame of the inflection point matching material is obtained by transitioning and fusing the transition reference frame and the starting frame of the inflection point matching material. Based on the connecting transition bridge and the inflection point blank time slot, the inflection point matching material is embedded into the preliminary material sequence to obtain the seamless reconstructed transient sequence of the preliminary material sequence; Based on the tendency label of the seamless recombination transient sequence and the target tendency path, the duration of the seamless recombination transient sequence is adjusted to obtain the harmonious material sequence of the preliminary material sequence.

7. The intelligent video automatic synthesis method based on multi-material fusion as described in claim 1, characterized in that, The step of adding time markers to each material in the harmonious material sequence based on the integrity boundary time of each material in the harmonious material sequence to obtain the time boundary material sequence of the harmonious material sequence includes: The multidimensional semantics of the harmonious material sequence are analyzed to obtain the boundary marker set of each material in the harmonious material sequence; The boundary marker set is conflict-resolved and merged to obtain the fusion boundary time of each material in the harmonious material sequence; Based on the content persistence of the fusion boundary moment, the fusion boundary moment is verified and corrected to obtain the candidate boundary moment of each material in the harmonious material sequence; Add time stamps to the candidate boundary moments and associate them with the original materials in the original media data to generate the time boundary material sequence of the harmonious material sequence.

8. The intelligent video automatic synthesis method based on multi-material fusion as described in claim 1, characterized in that, The step of defining a safe connection interval between adjacent materials based on the end markers of the preceding segment and the beginning markers of the following segment in the time boundary material sequence includes: The initial connection window of the adjacent materials is obtained by dividing the front end marker and the rear beginning marker of adjacent materials in the time boundary material sequence; Based on the end marker of the front segment and the beginning marker of the rear segment, the semantic fading point and semantic starting point in the adjacent material are located to obtain the semantic transition anchor point of the adjacent material; Based on the semantic transition anchor points of the materials, the boundaries of the initial connection window are migrated to obtain the safe connection interval of the adjacent materials.

9. The intelligent video automatic synthesis method based on multi-material fusion as described in claim 1, characterized in that, Based on the secure connection interval, the adjacent materials are transitioned and superimposed to obtain two segments of the adjacent materials with continuous protection video stream and continuous protection audio stream, including: Within the safe transition interval, the last frame sequence of the first segment of the adjacent material and the first frame sequence of the second segment of the adjacent material are pixel-weightedly synthesized frame by frame, and the screen switching rate is dynamically adjusted according to the severity of the difference in screen content between adjacent frames in the adjacent material to generate a transition video segment. Within the safe connection interval, the non-overlapping video segments of the preceding material, the transition video segment, and the non-overlapping video segments of the following material are spliced ​​together according to the time axis of the time boundary material sequence to obtain a continuous protection video stream; Extract the short-term energy envelope of the end of the preceding audio segment and the short-term energy envelope of the beginning of the following audio segment from the adjacent materials. Within the safe connection interval, perform linear energy attenuation on the preceding audio segment and simultaneously perform an equal amount of energy gain on the following audio segment to generate a continuous protection audio stream.

10. An intelligent video automatic synthesis system based on multi-source fusion, characterized in that, The system includes: Material Tag Extraction Module: Filters a multi-source material set from the original media data and extracts category tags and trend tags from the multi-source material set; Material type identification module: Based on the category tags, construct the material semantic spectrum of the materials in the multi-source material set, and based on the material semantic spectrum, identify the opposing material combinations and the remaining material set among the materials in the multi-source material set, and arrange the materials in the remaining material set into a preliminary material sequence; Material positioning and insertion module: Based on the tendency tags in the preliminary material sequence, generate the target tendency path of the preliminary material sequence, select inflection point matching materials from the opposing material combination that match the inflection point position of the target tendency path, and insert the inflection point matching materials into the preliminary material sequence according to the inflection point position of the target tendency path to obtain the harmonious material sequence; Time boundary marker module: Based on the integrity boundary time of each material in the harmonious material sequence, add time markers to each material in the harmonious material sequence to obtain the time boundary material sequence of the harmonious material sequence; Safe transition overlay module: Based on the end markers of the first segment and the beginning markers of the second segment of adjacent segments in the time boundary material sequence, the safe connection interval of the adjacent segments is defined, and the adjacent segments are overlaid based on the safe connection interval to obtain two segments of the adjacent segments of continuous protected video stream and continuous protected audio stream; Target video synthesis module: Extracts the audio dynamic signal of the continuous protection audio stream, and combines the audio dynamic signal to modulate and encapsulate the continuous protection video stream into a target video.

Images