Marketing video material automatic association analysis method based on deep learning

Abstract

The invention discloses a marketing video material automatic association analysis method based on deep learning, belongs to the technical field of multimedia information retrieval, and solves the problems that existing video material management depends on manpower, the efficiency is low, content understanding is one-sided, and intelligent matching is missing. According to the method, key frames and audio clips are extracted, visual and semantic features are extracted by adopting a convolutional neural network and a recurrent neural network and fused into multi-modal representation, automatic association and optimal matching of materials are realized based on similarity retrieval, clustering and time sequence analysis, and finally an intelligent retrieval chain crossing a material library is generated. And the effects of improving the material management efficiency and the creative output quality are achieved.

Description

Technical Field

[0001] This invention belongs to the field of multimedia information retrieval technology, and in particular relates to an automatic association analysis method for marketing video materials based on deep learning. Background Technology

[0002] In today's digital marketing landscape, video content has become a core medium for attracting user attention, and its management and utilization efficiency directly impacts brand communication and user interaction effectiveness. Video materials not only contain rich visual information but also integrate multiple elements such as sound and text. The key to improving marketing efficiency and the quality of creative output lies in how efficiently these vast resources can be mined and reused.

[0003] However, current video content management largely relies on manual operations, requiring significant manpower and time investment from classification and labeling to searching. Traditional methods struggle to fully understand the diversity of video content, often focusing only on single-modal information analysis and neglecting the inherent connection between visuals and sound. This results in the inability to fully explore the deeper semantic and emotional value of the content. For example, when searching for matching visuals and background music for a holiday promotional video, marketing teams have to spend a lot of time sifting through the entire content library, which is not only inefficient but also highly susceptible to subjective judgment biases or fragmented analysis of visuals and audio, leading to the loss of optimal creative combinations. This limitation is particularly pronounced when facing a large and continuously growing content library, causing creative work to become inefficient and repetitive, making it difficult to quickly respond to dynamic market demands.

[0004] A deeper technical challenge lies in the high complexity and heterogeneity of video content in terms of both visual and audio expression. Relying solely on analysis of one aspect cannot fully reconstruct the overall meaning of the video. Existing technologies lack intelligent mechanisms capable of simultaneously capturing and effectively associating visual elements with audio semantics, resulting in the inability to quickly and accurately locate content with similar themes or emotional expressions within massive amounts of material. Therefore, developing an analytical method that can automatically understand multimodal video content and achieve intelligent association and matching has become a core technical problem urgently needing to be solved in this field. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a deep learning-based method for automatic association analysis of marketing video materials, thereby resolving the issues present in the existing technologies.

[0006] Firstly, to achieve the above objectives, this invention provides an automatic association analysis method for marketing video materials based on deep learning, comprising the following steps: acquiring a target video sequence and extracting keyframe sequences and corresponding audio segments; extracting visual feature vectors from the keyframe sequences and semantic feature vectors from the audio segments, and fusing them into a multimodal feature representation; based on the multimodal feature representation, retrieving similar features from a material index database to form potential matching material groups; calculating the similarity between visual feature vectors and semantic feature vectors in the potential matching material groups, and determining highly associated material combinations based on a similarity threshold; extracting topic tags from the highly associated material combinations and performing clustering to obtain categorized material subsets; analyzing the temporal association patterns of images and sounds in the categorized material subsets and matching them with emotional expression templates to identify optimized matching pairs; and based on the optimized matching pairs, generating a cross-material library reuse path graph and constructing an intelligent retrieval chain.

[0007] Secondly, this invention also provides a deep learning-based automatic association analysis system for marketing video materials, used to implement a deep learning-based automatic association analysis method for marketing video materials. The system includes: a data acquisition module for acquiring target video sequences and extracting keyframe sequences and corresponding audio segments; a feature extraction module for extracting visual feature vectors of keyframe sequences and semantic feature vectors of audio segments, and fusing them into a multimodal feature representation; a similarity retrieval module for retrieving similar features from a material index database based on the multimodal feature representation to form potential matching material groups; a high-association determination module for calculating the similarity between visual feature vectors and semantic feature vectors in potential matching material groups, and determining high-association material combinations based on a similarity threshold; a topic clustering module for extracting topic tags from high-association material combinations and performing clustering to obtain categorized material subsets; a temporal analysis module for analyzing the temporal association patterns of images and sounds in the categorized material subsets and matching them with emotional expression templates to identify optimized matching pairs; and a path generation module for generating a cross-material library reuse path graph based on optimized matching pairs and constructing an intelligent retrieval chain.

[0008] Thirdly, the present invention also provides a computer terminal device, comprising: one or more processors; a memory coupled to the processors for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the deep learning-based marketing video material automatic association analysis method described in the first aspect above.

[0009] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the deep learning-based marketing video material automatic association analysis method described in the first aspect above.

[0010] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the deep learning-based marketing video material automatic association analysis method described in the first aspect above.

[0011] Compared with the prior art, the present invention has the following advantages and technical effects: This invention provides an automatic association analysis method for marketing video materials based on deep learning. Through multimodal feature fusion and intelligent matching mechanisms, this invention significantly improves the automation level and efficiency of video material management. It can deeply understand the intrinsic relationship between video visuals and sound, fully exploring the semantic and emotional value of the materials. Through multi-level similarity calculation and cluster analysis, it achieves accurate screening and theme classification of highly correlated material combinations. Based on temporal association patterns and emotional template matching optimization, it ensures a high degree of consistency in content and emotional expression of recommended materials. The final generated cross-material library intelligent retrieval chain provides a systematic reuse path for creative combinations, effectively improving the creation efficiency and content quality of marketing videos. Attached Figure Description

[0012] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is an overall flowchart of an embodiment of the present invention. Detailed Implementation

[0013] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0014] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0015] Example 1 like Figure 1 As shown, this embodiment provides a method for automatic association analysis of marketing video materials based on deep learning, including: Obtain the target video sequence and extract the keyframe sequence and corresponding audio segments; Visual feature vectors from keyframe sequences and semantic feature vectors from audio segments are extracted and fused into a multimodal feature representation. Based on multimodal feature representation, similar features are retrieved from the material index database to form potential matching material groups; Calculate the similarity between visual feature vectors and semantic feature vectors in potential matching material groups, and determine highly associated material combinations based on similarity thresholds; Extract topic tags from highly relevant material combinations and cluster them to obtain categorized material subsets; Analyze the temporal correlation patterns between images and sounds in the subset of classified materials, and match them with emotional expression templates to identify and optimize matching pairs; Based on optimized matching pairs, a reuse path graph across material libraries is generated, and an intelligent retrieval chain is constructed.

[0016] As one implementation method in this embodiment, the process of acquiring the target video sequence and extracting the keyframe sequence and corresponding audio segments includes: The target video sequence is obtained from the video material library, and frame sampling tools are used to parse it frame by frame. Each frame is timestamped to obtain a set of frame data. Based on the frame data set, image processing tools are used to evaluate the brightness and content changes of the frame images. If the change exceeds a preset threshold, it is marked as a key frame, and a key frame sequence is obtained. The audio track is segmented using audio processing tools, and the corresponding audio segments are obtained based on the time markers of the keyframe sequence. The volume of the audio segments is then determined to be up to standard. If it is not up to standard, noise reduction is performed. The keyframe sequence is time-aligned with the audio clip, and the two are integrated using a synthesis tool to determine the temporal consistency of the visual-audio data pair.

[0017] S101. Obtain the target video sequence from the video material library, and extract at least one keyframe sequence and corresponding audio segment using a preset frame sampling method to obtain an initial visual-audio data pair.

[0018] The target video sequence is obtained from a pre-set video material library. A frame sampling tool is used to analyze the video sequence frame by frame, and each frame is timestamped to obtain a set of time-stamped frame data. Based on the frame data set, an image processing tool is used to extract features from each frame, evaluating the brightness and content changes. If the change exceeds a preset threshold, it is marked as a keyframe, resulting in a keyframe sequence. An audio processing tool is used to segment the audio track of the target video sequence. For each time stamp in the keyframe sequence, a corresponding audio segment is obtained. The volume of the audio segment is checked; if not, noise reduction is performed to obtain an optimized audio segment. The keyframe sequence and the optimized audio segment are time-aligned, and a synthesis tool is used to integrate them, ensuring the temporal consistency of the integrated visual-audio data pair.

[0019] For example, when processing video footage, suppose a 30-second advertisement video is selected as the target video sequence from a pre-defined video library. A frame sampling tool is used to analyze this video frame by frame, extracting images at a rate of 30 frames per second, resulting in 900 frames of data. Each frame is then timestamped, forming a set of time-stamped frame data. This process ensures that each frame accurately corresponds to a specific time point in subsequent processing, laying the foundation for keyframe extraction and audio-visual alignment.

[0020] In one possible implementation, feature extraction is performed on the frame data set, and image processing tools are used to analyze the brightness values ​​and content changes of each frame.

[0021] For example, by comparing the differences in pixel grayscale values ​​and structural similarities between adjacent frames, a comprehensive threshold for brightness and content changes is set. If a frame experiences a significant jump in brightness and content, such as a scene switching from a dark room to a bright outdoor panorama, that frame is marked as a keyframe, ultimately forming a keyframe sequence. Subsequently, the audio track of the target video is segmented, and the corresponding audio segments are extracted based on the time markers of the keyframes. Assuming a keyframe is at 10 seconds, the corresponding audio segment within its time window is extracted, and its volume is checked to see if it meets a preset standard. If the volume is insufficient, background noise is removed using noise reduction tools, and gain processing is performed to obtain an optimized audio segment. This processing significantly improves audio clarity, providing high-quality data for subsequent analysis.

[0022] In one possible implementation, the keyframe sequence is time-aligned with the optimized audio clip, and the two are integrated using a synthesis tool.

[0023] For example, ensure that the start time error between the keyframe at the 10-second mark and the corresponding audio segment is less than 0.1 seconds. Then verify the timing consistency of the integrated visual and audio data to ensure that the scene transitions and sound changes are synchronized. This alignment method effectively avoids audio-visual asynchrony and improves the overall video quality.

[0024] For example, in the application of keyframe sequences, suppose in an advertising video, keyframes capture the transformation of a product from packaging to use. Combined with aligned audio narration, this can more intuitively convey the product's value and enhance viewer recall. This technical effect is particularly important in marketing scenarios, as it can improve user engagement and brand awareness.

[0025] As one implementation method in this embodiment, the process of extracting the visual feature vectors of the keyframe sequence and the semantic feature vectors of the audio segments, and fusing them into a multimodal feature representation includes: Convolutional neural networks are used to process keyframe sequences to extract visual feature vectors, while recurrent neural networks are used to process audio segments to extract semantic feature vectors. For visual feature vectors and semantic feature vectors, a fusion processing tool is used to perform weighted combination to obtain fused multimodal feature representation data; If the fused multimodal feature representation data does not reach the preset threshold range, the multimodal feature representation data is optimized using an adjustment tool to obtain the adjusted feature representation data. Based on the adjusted feature representation data, a classification tool is used to classify the data into categories, determine whether they meet the preset multimodal feature standards, and obtain the final multimodal feature classification results.

[0026] S102. Based on the initial visual and audio data pair, a convolutional neural network is used to process the keyframe sequence to extract visual feature vectors, while a recurrent neural network is used to process the audio segments to extract semantic feature vectors, thus determining the fused multimodal feature representation.

[0027] Based on initial visual and audio data, a feature extraction tool is used to process the keyframe sequence to obtain the visual feature vector. Simultaneously, a semantic analysis tool is used to process the audio segment to obtain the semantic feature vector, thus determining the preliminary representation results of the two feature vectors. The visual and semantic feature vectors are then weighted and combined using a fusion processing tool to obtain the fused multimodal feature representation data. If the fused multimodal feature representation data does not reach a preset threshold range, an adjustment tool is used to optimize the multimodal feature representation data, obtaining the adjusted feature representation data. Based on the adjusted feature representation data, a classification tool is used to classify the data and determine whether it meets the preset multimodal feature criteria, obtaining the final multimodal feature classification result.

[0028] For example, in a short video showcasing a product, the main subjects are the product and the presenter. Feature extraction tools analyze the appearance of the product, the presenter's actions, and scene transitions to generate corresponding visual feature vectors. This data will serve as the basis for subsequent fusion.

[0029] In one possible implementation, semantic analysis of audio segments can be performed using semantic analysis tools to identify keywords and sentiment in the audio. Assuming the audio segment is a dialogue, the tool might extract keywords such as "happy" and "celebrating," determine the sentiment as positive, and generate a semantic feature vector. This vector generation facilitates subsequent matching with visual features, ensuring content consistency between the two.

[0030] For example, for the fusion of visual feature vectors and semantic feature vectors, a weighted combination tool can be used to assign appropriate weights to features of different modalities to obtain fused multimodal feature representation data.

[0031] In one possible implementation, the adjusted feature representation data is fed into a classification tool for categorization. Assuming the preset multimodal feature criteria are sentiment consistency and content relevance, the classification tool will determine whether the adjusted data fits the category of "positive sentiment + dynamic visuals." If the visuals depict a cheerful dance and the audio sentiment is also positive, the classification result is considered to meet the criteria. This classification aids in subsequent content filtering, ensuring that the final multimodal data is highly consistent in both sentiment and content.

[0032] For example, in the optimization process, if the fused data is still not ideal, the visual data can be re-extracted by adjusting the tools, focusing on more discriminative elements in the image, or more refined sentiment analysis can be performed on the audio clips to improve the discriminativeness and consistency of the multimodal feature representation.

[0033] In one possible implementation, every step of the process revolves closely around the processing of video footage, ensuring the synergy between visuals and audio. For example, when extracting keyframe sequences, frames with significant changes in visual content are prioritized, while in audio processing, the focus is on sound segments that correspond to the changes in visuals. This correspondence-based processing effectively improves the usability of the final data, providing higher-quality material support for content creation.

[0034] As one implementation method in this embodiment, the process of retrieving similar features from the material index database based on multimodal feature representation to form a potential matching material group includes: Multimodal feature representation data is obtained from the material index database and compared with the features stored in the database. The similarity value between features is calculated using the Euclidean distance calculation tool to obtain a similarity distribution result set. Based on the similarity distribution result set, the similarity value of each feature is compared with a preset threshold. If the similarity value exceeds the preset threshold, it is judged as a potential matching feature group, and a preliminary potential matching feature set is determined. For the initial set of potential matching features, obtain the material metadata corresponding to each potential matching feature group, and verify the context information of the material through a metadata comparison tool to obtain a verified subset of matching features; Based on the validated subset of matching features, a clustering tool is used to group the features within the subset to determine the final matching material group results.

[0035] S103. For the fused multimodal feature representation, obtain the distribution of similar features in the pre-established material index database. If the distribution of similar features exceeds a preset threshold, it is judged as a potential matching material group, and a preliminary matching result set is obtained.

[0036] Multimodal feature representation data after fusion is obtained from a pre-established material index database. This data is compared with features stored in the database, and the similarity value between two features is calculated using an Euclidean distance calculation tool, resulting in a similarity distribution set for each feature. Based on this similarity distribution set, the similarity value of each feature is compared with a preset threshold. If the similarity value exceeds the preset threshold, it is identified as a potential matching feature group, thus determining the preliminary potential matching feature set. For this preliminary potential matching feature set, the material metadata corresponding to each potential matching feature group is obtained. The context information of the material is verified using a metadata comparison tool, resulting in a verified matching feature subset. Based on the verified matching feature subset, a clustering tool is used to group the features within the subset, determining the final matching material group results.

[0037] For example, when processing fused multimodal feature representation data, relevant data can be extracted from a pre-established material index database for comparison. Suppose the target is a video with a "holiday promotion" atmosphere, whose feature representation data incorporates festive visual elements (such as red decorations and gifts) and cheerful audio features. Using a comparison tool, the Euclidean distance between the target features and the material features in the database is calculated. If the distance value is less than a preset similarity threshold, the material is identified as a potential match. This method can quickly identify materials with a style and theme similar to the target content.

[0038] For example, the filtering logic can be further refined when processing the similarity distribution result set.

[0039] In one possible implementation, if a feature has a similarity value of 2.8, exceeding the threshold of 3.0, it is included in the initial potential matching feature set. Assuming 10 features enter this set, it's observed that some features may be similar in visual style but differ significantly in audio emotion. Therefore, further metadata validation is needed to eliminate inconsistent matches. The contextual information of the metadata includes the creation background of the material, time tags, etc. For example, if the background music of a material is labeled "sad," while the target feature's audio emotion is "cheerful," then that material will be removed. This validation method helps improve the accuracy of the matching.

[0040] For example, after obtaining a validated subset of matching features, clustering tools can be used to group these features. Suppose the validated subset contains 5 features, with 3 features leaning towards a "retro" visual style and 2 leaning towards a "modern" style. The clustering tool will divide them into two groups. After grouping, the final matching material group results can provide a basis for subsequent content recommendation or classification.

[0041] For example, materials from the "Retro" group might be used for videos with a nostalgic theme, while those from the "Modern" group are suitable for tech-themed content. This grouping makes the application scenarios for the materials clearer.

[0042] In one possible implementation, the verification process for material metadata can incorporate multi-dimensional comparisons.

[0043] For example, validation isn't limited to emotional tone; it can also include information like the duration of the footage and its resolution. If the target feature corresponds to a 30-second video, but a matching clip is 5 minutes long, it might be rejected due to incompatibility. This multi-dimensional validation further enhances the usability of the matching results, providing higher-quality data support for subsequent content processing.

[0044] For example, the process of determining the final matching material group can also incorporate a manual review mechanism. After verification and clustering, if there are still a few disputed feature groups, such as a feature being ambiguous between "retro" and "modern" styles, its classification can be confirmed through manual annotation. Although this method increases the workload slightly, it can significantly improve the accuracy of the matching results, especially when dealing with complex multimodal features.

[0045] In one possible implementation, the above comparison, verification, and clustering processes can also dynamically adjust the thresholds and weights.

[0046] For example, the initial threshold is 3.0, but if too few matching results are found in practical applications, it can be appropriately relaxed to 3.5 to include more potential matching features. This flexible adjustment can adapt to the needs of different scenarios, ensuring that the matching results are neither too broad nor too narrow, providing a wider scope for the application of multimodal features.

[0047] As one implementation method in this embodiment, the process of calculating the similarity between visual feature vectors and semantic feature vectors in a potential matching material group, and determining highly associated material combinations based on a similarity threshold, includes: The visual feature vector and semantic feature vector are obtained from each matching material group, and the cosine similarity value between the two is calculated to obtain the similarity result of the matching material group. Based on the similarity results, a preset threshold is used for comparison. If the similarity result is higher than the preset threshold, the matched material group is determined to be a highly related material combination and marked as a candidate combination to be processed. For candidate combinations, the distribution data of visual feature vectors and semantic feature vectors are obtained. The consistency of the distribution data is checked by a feature comparison tool to determine whether there are abnormal distributions. If there are abnormalities, the distribution data is corrected to obtain the corrected feature data. Based on the corrected feature data, clustering tools are used to group highly correlated material combinations to determine the final classification results.

[0048] S104. Using the preliminary matching result set, calculate the cosine similarity between the visual feature vector and the semantic feature vector within each matched material group. If the cosine similarity is higher than the threshold, it is determined to be a highly associated material group.

[0049] Using a pre-established feature extraction tool, visual and semantic feature vectors are obtained from each matching material group. The cosine similarity value between the two vectors is calculated for each matching material group to obtain the similarity result. Based on the similarity result, a preset threshold is used for comparison. If the similarity result is higher than the preset threshold, the matching material group is determined to be a highly correlated material combination and marked as a candidate combination to be processed. For the candidate combination, the distribution data of the visual and semantic feature vectors are obtained. The consistency of the distribution data is checked using a feature comparison tool to determine if there are any abnormal distributions. If abnormalities are found, the distribution data is corrected to obtain corrected feature data. Based on the corrected feature data, a clustering tool is used to group the highly correlated material combinations to determine the final classification result.

[0050] For example, in business scenarios involving multimodal content matching, the extraction and comparison of visual and semantic feature vectors can be approached from the perspective of content analysis to explore how to ensure matching accuracy. Suppose we are dealing with a video content library, and the goal is to select segments highly relevant to the target video content. For visual feature vector extraction, a pre-trained image recognition model can be used to decompose video frames into multiple keyframes, extracting features such as color distribution and texture information from each keyframe to form a high-dimensional vector, such as a 512-dimensional vector representation. Semantic feature vectors can be extracted using natural language processing tools from video subtitles or speech-transcribed text to form another vector representation, also such as a 512-dimensional vector.

[0051] For example, when calculating cosine similarity, the cosine of the angle between the visual feature vector and the semantic feature vector is used as the metric. A comprehensive similarity result is obtained by calculating the ratio of the dot product of the two vectors to their magnitude. If this result is higher than a preset similarity threshold, the set of materials is considered a highly relevant combination. For instance, a material depicting a "family dinner" has a high similarity to its semantic features "warmth" and "reunion," and can be accurately selected.

[0052] This method can effectively filter out materials with consistent content.

[0053] For example, to verify the consistency of feature distribution data, anomalies can be identified by analyzing the distribution ranges of visual and semantic feature vectors. Suppose that in a candidate combination, the distribution of visual feature vectors is concentrated between 0.7 and 0.9, while the distribution of semantic feature vectors is concentrated between 0.2 and 0.4. This significant inconsistency may indicate a bias in the data extraction process, such as subtitles not accurately reflecting the video content. In this case, the semantic feature vectors can be corrected, for example, by re-extracting text data or adjusting weights, to obtain corrected distribution data that more closely approximates the distribution range of visual features. This correction helps improve the reliability of subsequent classification.

[0054] For example, in determining the final classification results, clustering tools can be used to group highly relevant material combinations, based on the distance between feature vectors. Suppose there are multiple highly relevant material combinations; clustering tools can be used to divide them into several categories, each representing a marketing theme or style, such as "product showcase," "user testimonials," and "brand story."

[0055] For example, in business scenarios, this multimodal feature matching process can be extended to more granular applications, such as filtering advertising clips from a video content library. Through comprehensive analysis of visual and semantic features, advertising content that aligns with the brand's tone can be quickly identified. This extended application further demonstrates the importance of feature comparison and classification in content management, significantly improving the efficiency and accuracy of content filtering.

[0056] As one implementation method in this embodiment, the process of extracting topic tags from highly correlated material combinations and performing clustering to obtain a subset of categorized materials includes: We retrieve highly relevant material combinations from the material database and extract the corresponding theme tags. We then use a keyword extraction tool to perform preliminary sorting of the theme tags, resulting in a preliminary categorized set of tags. Based on the initially classified tag set, a clustering tool is used to group the tags. If the semantic similarity of a tag with other tags exceeds a preset threshold, it is grouped into the same group to determine the tag category after grouping. From the grouped label categories, obtain the material combination subset corresponding to each category, and use the feature matching tool to verify the correlation of the material data in the subset, determine whether the material in the subset meets the high correlation standard, and obtain the verified material subset; After verifying the material subsets, data integration tools are used to bind the subsets with the corresponding theme tag categories, generating the final categorized material subset set and determining the categorization result.

[0057] S105. Obtain the topic tags in the highly relevant material combination, and use the clustering method to group the topic tags to obtain the classified material subsets.

[0058] By using a pre-established material database, highly relevant material combinations are acquired. For each material combination, corresponding theme tags are extracted. Keyword extraction tools are used to initially organize these theme tags, resulting in a preliminary tag set. Based on this preliminary tag set, a clustering tool is used to group the tags. If the semantic similarity of a tag to other tags exceeds a preset threshold, it is grouped into the same group, thus determining the tag category. From these grouped tag categories, a subset of material combinations corresponding to each category is obtained. For the material data within each subset, a feature matching tool is used to verify its relevance, determining whether the materials within the subset meet the high relevance criteria, resulting in a verified material subset. Using this verified material subset, a data integration tool is used to bind the subset to its corresponding theme tag category, generating a final categorized material subset set, thus determining the classification result.

[0059] For example, when processing highly relevant material combinations in a material database, you can first extract material data that is highly relevant to a specific marketing theme. Suppose the focus is on the theme of "outdoor sports," the database might contain images and videos related to hiking, running, and cycling. Through a pre-established indexing mechanism, the system will filter out material combinations highly relevant to the theme of "sports and vitality," such as image sets containing elements of "hiking equipment" and "running scenes." These material combinations will serve as the basis for subsequent analysis.

[0060] For example, based on the extracted topic tags, a keyword extraction tool can be used to organize tags related to "beauty and skincare." Assuming the extracted tags include "lipstick," "foundation," "skincare products," and "makeup tutorials," the tool will initially categorize these tags according to semantic relevance, forming a preliminary tag set. For instance, "lipstick" and "foundation" might be categorized under "makeup," while "skincare products" might be categorized under "basic skincare." This process aims to provide a clear semantic basis for subsequent grouping.

[0061] For example, when using clustering tools to group tags, a semantic similarity threshold can be set. If the semantic similarity between "running" and "fitness" exceeds the threshold, they will be grouped into the same category, "sports and health." This grouping method helps to structure complex tag data, facilitating the subsequent classification and management of materials.

[0062] For example, after obtaining the subset of materials for each category, a feature matching tool can be used to verify the relevance of the subset of materials under the "Sports and Health" category. Suppose the subset contains multiple images, most of which have visual features highly consistent with the sports theme, while a few images lean towards other themes. After verification, these inconsistent images may be removed, ensuring high relevance of the materials within the subset. This step effectively improves the accuracy of material classification.

[0063] For example, when binding a validated subset of materials to topic tag categories, data integration tools can be used to generate the final classification results. Suppose the "makeup" category ultimately binds to multiple highly relevant images, forming a separate subset of materials. This process not only achieves accurate matching between materials and tags but also provides structured data support for subsequent content recommendation or display, helping to improve user experience and content management efficiency.

[0064] For example, the processing methods for different topic tag categories can be flexibly adjusted throughout the process. If a new category, "Ocean Scenery," is added, the system can repeat the above process, from material extraction to category binding, ensuring that the materials under the new category also have high relevance. This scalable design can adapt to constantly changing content needs, ensuring the integrity and usability of the classification system.

[0065] S106. For the categorized material subset, extract the temporal correlation pattern of the video and audio in the subset. If the temporal correlation pattern matches the preset emotional expression template, it is judged as an optimized matching pair, and the final recommended material list is obtained.

[0066] Temporal data of visuals and audio are extracted from the categorized material subsets. For each set of visuals and audio segments, feature points on the timeline are obtained, and a pre-established temporal comparison tool is used for matching analysis to obtain the temporal association pattern results of the visuals and audio. Based on the temporal association pattern results, a sentiment expression template database is used for comparison. If the matching degree between the association pattern and the preset sentiment expression template exceeds a preset threshold, the visuals and audio are determined as preliminary optimized matching pairs, and a preliminary matching pair list is established. For each set of data in the preliminary matching pair list, the detailed features of the sentiment expression template are obtained, and the sentiment consistency of the visuals and audio is analyzed using a feature comparison tool to obtain sentiment consistency score data. Based on the sentiment consistency score data, if the score is higher than a preset threshold, the visuals and audio are determined as final optimized matching pairs, and a sorting tool is used to prioritize them to obtain the final recommended material list.

[0067] For example, when processing categorized subsets of materials, one can start by analyzing the temporal data of visuals and audio, examining key features along the timeline. Temporal data refers to the patterns of change in visuals and audio over time, such as the moment a product makes a stunning appearance in the visuals or the points in the audio where product selling points are emphasized and the background music reaches its climax. Suppose a subset of materials contains a product advertisement video where the transition from a close-up to a full view of the product occurs at the 8-second mark, and the narration and sound effects emphasizing the product's core advantages appear at the same time. Using a pre-established temporal comparison tool, the overlap of these two key features can be identified, thus revealing the correlation pattern between visuals and audio in terms of content emphasis.

[0068] For example, comparing the results of temporal correlation patterns with the sentiment expression template database can be understood as a sentiment matching process. The sentiment expression template database stores various preset sentiment patterns applicable to marketing scenarios. For instance, the "pleasure" template might include a combination of smiling faces and bright colors in the image, and a light rhythm and positive tone in the sound. Suppose a piece of material shows a consumer showing a satisfied smile after using a product, the scene is bright, and the background music is light and lively. This has a high match degree with the "pleasure" template, exceeding a preset threshold, and can then be classified as a preliminary optimized match.

[0069] For example, when analyzing the emotional consistency of the initial matching pair list, feature comparison tools can be used to delve deeper into the detailed features of the visuals and sounds. For instance, bright colors and radiant smiles in the visuals, along with a light and cheerful tone and upbeat background music, highly match the detailed features of the "pleasant" template. The emotional consistency score exceeds the preset threshold, thus it is judged as a final optimized matching pair. This meticulous comparison ensures the consistency of the emotional expression in the materials.

[0070] For example, during the generation of the final recommended content list, the sorting tool can prioritize content based on its sentiment consistency score. Assuming there are three final optimized match pairs with scores of 85, 82, and 78 respectively, the tool will prioritize the content with the score of 85. This sorting method intuitively presents the content that best matches emotional needs, facilitating subsequent selection and application.

[0071] For example, the process of analyzing the temporal correlation between visuals and sound, matching emotional templates, and maintaining consistency scores collectively constructs a screening system that progresses from coarse to fine. The temporal correlation model provides initial points of convergence, emotional template matching further clarifies the emotional tone, and the consistency score ensures a high degree of uniformity in detail. This progressive approach effectively improves the accuracy of the combined materials in expressing emotions, providing high-quality foundational material for subsequent content creation.

[0072] For example, in practical implementation, suppose a subset of footage is used for short video production. If a person in the video smiles at the 5-second mark, and laughter also appears in the audio at the 5-second mark, a high correlation is identified using a time-series comparison tool. Subsequently, the footage achieves a 90% match with the "joyful" template and a high emotional consistency score of 88, ultimately being prioritized for recommendation. This multi-dimensional analysis ensures the footage's consistency in time, emotion, and detail, providing strong support for creative work.

[0073] S107. Based on the final recommended material list, generate a reuse path diagram across the material library, and determine the connection strength between nodes in the path diagram to form an intelligent retrieval chain.

[0074] Based on the final recommended content list across different content libraries, the identifier information and category tag of each content library are obtained. By comparing the distribution of similar tags in different content libraries, the initial association strength between the content libraries is determined, resulting in initial path graph node connection data. For this initial path graph node connection data, a preset threshold is used to filter the connection strength. If the connection strength is lower than the preset threshold, the connection is removed, resulting in a simplified path graph node connection set. From this simplified set, the number and strength distribution of connections for each node are obtained. Common graph traversal tools are used to calculate the multi-path dependencies between the nodes, determining the coreity of each node in the path graph and obtaining a ranking list of core nodes. Based on this core node ranking list, for the top-ranked nodes, common proximity search tools are used to construct priority paths for the intelligent retrieval chain. By connecting the core nodes with strongly associated paths to surrounding nodes, the final cross-content library reuse path graph structure is determined.

[0075] For example, when processing a recommended content list across different content libraries, one can start by analyzing the identifiers and category tags of each content item to understand the distribution of similar tags across different libraries. Suppose there is a video content library and an audio content library. One video item is tagged "Holiday Promotion," and another audio item is tagged "Happy Holiday." By comparing the semantic commonalities of the tags, a preliminary judgment can be made that these two items may be related. The initial association strength is then calculated to be 0.75, and this is included in the initial path graph node connection data. This approach helps to quickly filter out potential cross-library matching combinations.

[0076] In one possible implementation, the threshold for connection strength in the initial path graph node connection data can be set to 0.6. If the connection strength between two materials is only 0.5, then that connection is removed, and connections with higher strength are retained, ultimately forming a simplified set of path graph node connections.

[0077] For example, if a video clip has association strengths of 0.7, 0.55, and 0.8 with three clips in an audio library, after filtering, only the links with strengths of 0.7 and 0.8 are retained. This filtering mechanism effectively reduces redundant data and improves the efficiency of subsequent analysis.

[0078] For example, for a simplified path graph node connection set, the core importance of each node can be determined by analyzing the number and strength distribution of its connections. Suppose a video clip node connects to five audio clip nodes with an average strength of 0.85, while another node connects to only two nodes with an average strength of 0.65. The former is considered the more core node and is prioritized in the sorting list. This sorting method highlights the crucial role of important materials in cross-library reuse, providing a basis for subsequent path construction.

[0079] In one possible implementation, for the sorted list of core nodes, a proximity search tool can be used to construct a priority path for the intelligent retrieval chain of the nodes that rank higher.

[0080] For example, select the top three core nodes and search for the strongest related nodes around them to form a strong connection path. Assuming the connection strengths between core node A and nodes B and C are 0.9 and 0.88 respectively, prioritize connecting A to B and C, forming a reuse path from video to audio. This path design ensures high relevance when reusing materials and improves the overall coordination of the combination.

[0081] For example, the final cross-media reuse path diagram structure can be determined based on the aforementioned priority paths, connecting core nodes with strongly related surrounding nodes. Assuming that in the final path diagram, core node A is connected to D via B, forming a complete video-audio-background sound effect reuse chain. This structured path diagram provides clear guidance for the rapid retrieval of cross-media materials, while ensuring emotional or stylistic consistency between materials, thus contributing to improved overall content creation quality and efficiency.

[0082] Based on this, this invention provides an automatic association analysis method for marketing video materials based on deep learning. This invention significantly improves the automation level and efficiency of video material management through multimodal feature fusion and intelligent matching mechanisms. This invention can deeply understand the intrinsic relationship between video visuals and sound, fully exploring the semantic and emotional value of the materials. Through multi-level similarity calculation and cluster analysis, it achieves accurate screening and theme classification of highly correlated material combinations. Based on temporal association patterns and emotional template matching optimization, it ensures a high degree of consistency in content and emotional expression of recommended materials. The final generated cross-material library intelligent retrieval chain provides a systematic reuse path for creative combinations, effectively improving the creation efficiency and content quality of marketing videos.

[0083] Example 2 In this embodiment, a computer terminal device is provided, including: one or more processors; a memory coupled to the processors for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the above-described method for automatic association analysis of marketing video materials based on deep learning.

[0084] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-described method for automatic association analysis of marketing video materials based on deep learning.

[0085] In this embodiment, an electronic device is also provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the above-described method for automatic association analysis of marketing video materials based on deep learning.

[0086] In this embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the above-described method for automatic association analysis of marketing video materials based on deep learning.

[0087] The aforementioned program can run on a processor or be stored in memory (or a computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0088] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented by different modules for different steps.

[0089] This embodiment provides such an apparatus or system. The system, referred to as a deep learning-based automatic association analysis system for marketing video content, includes: The data acquisition module is used to acquire the target video sequence and extract the keyframe sequence and corresponding audio segments; The feature extraction module is used to extract visual feature vectors from keyframe sequences and semantic feature vectors from audio segments, and fuse them into a multimodal feature representation. The similarity retrieval module is used to retrieve similar features from the material index database based on multimodal feature representation to form potential matching material groups; The high correlation determination module is used to calculate the similarity between visual feature vectors and semantic feature vectors in potential matching material groups, and to determine high correlation material combinations based on similarity thresholds; The topic clustering module is used to extract topic tags from highly related material combinations and perform clustering to obtain categorized material subsets; The temporal analysis module is used to analyze the temporal correlation patterns between images and sounds in the categorized material subset and match them with the emotional expression template to identify and optimize matching pairs; The path generation module is used to generate reusable path graphs across material libraries based on optimized matching pairs and to build intelligent search chains.

[0090] The system or apparatus is used to implement the functions of the methods in the above embodiments. Each module in the system or apparatus corresponds to each step in the method, as has been described in the method and will not be repeated here.

[0091] The above implementation method solves the problem of automatic association analysis of marketing video materials based on deep learning in related technologies, thereby ensuring that the problems existing in the prior art are resolved.

[0092] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for automatic association analysis of marketing video materials based on deep learning, characterized in that, Includes the following steps: Obtain the target video sequence and extract the keyframe sequence and corresponding audio segments; Visual feature vectors from keyframe sequences and semantic feature vectors from audio segments are extracted and fused into a multimodal feature representation. Based on multimodal feature representation, similar features are retrieved from the material index database to form potential matching material groups; Calculate the similarity between visual feature vectors and semantic feature vectors in potential matching material groups, and determine highly associated material combinations based on similarity thresholds; Extract topic tags from highly relevant material combinations and cluster them to obtain categorized material subsets; Analyze the temporal correlation patterns between images and sounds in the subset of classified materials, and match them with emotional expression templates to identify and optimize matching pairs; Based on optimized matching pairs, a reuse path graph across material libraries is generated, and an intelligent retrieval chain is constructed.

2. The method according to claim 1, characterized in that, The process of acquiring the target video sequence and extracting the keyframe sequence and corresponding audio segments includes: The target video sequence is obtained from the video material library, and frame sampling tools are used to parse it frame by frame. Each frame is timestamped to obtain a set of frame data. Based on the frame data set, image processing tools are used to evaluate the brightness and content changes of the frame images. If the change exceeds a preset threshold, it is marked as a key frame, and a key frame sequence is obtained. The audio track is segmented using audio processing tools, and the corresponding audio segments are obtained based on the time markers of the keyframe sequence. The volume of the audio segments is then determined to be up to standard. If it is not up to standard, noise reduction is performed. The keyframe sequence is time-aligned with the audio clip, and the two are integrated using a synthesis tool to determine the temporal consistency of the visual-audio data pair.

3. The method according to claim 1, characterized in that, The process of extracting visual feature vectors from keyframe sequences and semantic feature vectors from audio segments, and fusing them into a multimodal feature representation, includes: Convolutional neural networks are used to process keyframe sequences to extract visual feature vectors, while recurrent neural networks are used to process audio segments to extract semantic feature vectors. For visual feature vectors and semantic feature vectors, a fusion processing tool is used to perform weighted combination to obtain fused multimodal feature representation data; If the fused multimodal feature representation data does not reach the preset threshold range, the multimodal feature representation data is optimized using an adjustment tool to obtain the adjusted feature representation data. Based on the adjusted feature representation data, a classification tool is used to classify the data into categories, determine whether they meet the preset multimodal feature standards, and obtain the final multimodal feature classification results.

4. The method according to claim 1, characterized in that, The process of retrieving similar features from the material index database based on multimodal feature representation to form potential matching material groups includes: Multimodal feature representation data is obtained from the material index database and compared with the features stored in the database. The similarity value between features is calculated using the Euclidean distance calculation tool to obtain a similarity distribution result set. Based on the similarity distribution result set, the similarity value of each feature is compared with a preset threshold. If the similarity value exceeds the preset threshold, it is judged as a potential matching feature group, and a preliminary potential matching feature set is determined. For the initial set of potential matching features, obtain the material metadata corresponding to each potential matching feature group, and verify the context information of the material through a metadata comparison tool to obtain a verified subset of matching features; Based on the verified subset of matching features, a clustering tool is used to group the features within the subset to determine the final matching material group results.

5. The method according to claim 1, characterized in that, The process of calculating the similarity between visual feature vectors and semantic feature vectors in a potential matching material group, and determining highly associated material combinations based on a similarity threshold, includes: The visual feature vector and semantic feature vector are obtained from each matching material group, and the cosine similarity value between the two is calculated to obtain the similarity result of the matching material group. Based on the similarity results, a preset threshold is used for comparison. If the similarity result is higher than the preset threshold, the matched material group is determined to be a highly related material combination and marked as a candidate combination to be processed. For candidate combinations, the distribution data of visual feature vectors and semantic feature vectors are obtained. The consistency of the distribution data is checked by a feature comparison tool to determine whether there are abnormal distributions. If there are abnormalities, the distribution data is corrected to obtain the corrected feature data. Based on the corrected feature data, clustering tools are used to group highly correlated material combinations to determine the final classification results.

6. The method according to claim 1, characterized in that, The process of extracting topic tags from highly correlated material combinations and performing clustering to obtain categorized material subsets includes: We retrieved highly relevant material combinations from the material database and extracted the corresponding theme tags. We then used a keyword extraction tool to perform preliminary sorting of the theme tags, resulting in a preliminary categorized set of tags. Based on the initially classified tag set, a clustering tool is used to group the tags. If the semantic similarity of a tag with other tags exceeds a preset threshold, it is grouped into the same group to determine the tag category after grouping. From the grouped tag categories, obtain the material combination subset corresponding to each category, and use the feature matching tool to perform correlation verification on the material data in the subset to determine whether the materials in the subset meet the high correlation standard, and obtain the verified material subset; After verifying the material subsets, data integration tools are used to bind the subsets with the corresponding theme tag categories, generating the final categorized material subset set and determining the categorization result.

7. A deep learning-based automatic association analysis system for marketing video materials, characterized in that, The system for implementing the method of any one of claims 1-6 comprises: The data acquisition module is used to acquire the target video sequence and extract the keyframe sequence and corresponding audio segments; The feature extraction module is used to extract visual feature vectors from keyframe sequences and semantic feature vectors from audio segments, and fuse them into a multimodal feature representation. The similarity retrieval module is used to retrieve similar features from the material index database based on multimodal feature representation to form potential matching material groups; The high correlation determination module is used to calculate the similarity between visual feature vectors and semantic feature vectors in potential matching material groups, and to determine high correlation material combinations based on similarity thresholds; The topic clustering module is used to extract topic tags from highly related material combinations and perform clustering to obtain categorized material subsets; The temporal analysis module is used to analyze the temporal correlation patterns between images and sounds in the categorized material subset and match them with the emotional expression template to identify and optimize matching pairs; The path generation module is used to generate reusable path graphs across material libraries based on optimized matching pairs and to build intelligent search chains.

8. A computer terminal device, characterized in that, include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

Images