A content extraction and delivery method based on multi-modal business order video

Abstract

The application relates to the technical field of big data analysis, and particularly discloses a content extraction and delivery method based on multi-modal business order videos, which comprises the following steps: calling a preset domain name-brand mapping library to output a preliminary brand identification or extract to-be-verified brand information and obtain directional multi-modal features; determining a core brand based on the preliminary brand identification or the directional multi-modal features, the to-be-verified brand information and a brand vector library, and performing standardization processing to obtain a standardized brand identification; based on the standardized brand identification, combining platform characteristics and target user portrait data of the business order video, calling a delivery strategy library to generate a directional delivery strategy of an adaptive brand type, and pushing the business order video to a target user group according to the directional delivery strategy, while monitoring delivery effect indexes in real time to obtain monitoring data, adjusting current multi-dimensional weights according to the monitoring data, and generating an effect report; the brand identification accuracy in the business order video is improved, and the delivery strategy can be continuously optimized.

Description

Technical Field

[0001] This invention relates to the field of big data analytics, and in particular to a method for content extraction and delivery based on multimodal commercial video. Background Technology

[0002] In today's digital marketing era, commercial video ads, as an important advertising format, are widely disseminated on short video and social media platforms. With the diversification of brand promotion needs and the increasing richness of video content, accurately extracting brand information from commercial videos and achieving precise targeting has become crucial. Multimodal commercial video content extraction and targeting methods meet the market's demand for efficient and precise marketing. With the continued development of short video and social media platforms, and the increasingly fierce brand competition, this multimodal content extraction and targeting method has broad application prospects and is expected to become one of the core technologies in the future digital marketing field, driving the industry towards a more intelligent and precise direction.

[0003] However, existing commercial video processing technologies primarily rely on text keyword matching. This single-modal analysis method lacks an effective mechanism for distinguishing colloquial expressions and polysemous words such as "geely" and "ideal." Strict rules lead to a decrease in recognition rate, while lenient rules result in a sharp drop in accuracy. Traditional methods suffer from high false positive rates, making it difficult to identify the true advertiser. Furthermore, existing technologies fail to adequately integrate and utilize multimodal complementary cues such as video visuals (logo / scene), audio (brand colloquial mentions), and cover images, resulting in insufficient feature completeness and an inability to comprehensively and accurately extract brand information. In the ad delivery phase, the failure to fully leverage platform characteristics and target user profiles for precise targeting impacts ad performance, and the lack of real-time monitoring and dynamic adjustment mechanisms prevents timely optimization of strategies based on ad performance.

[0004] Therefore, this invention proposes a method for content extraction and delivery based on multimodal commercial video. Summary of the Invention

[0005] This invention provides a method for content extraction and delivery based on multimodal commercial video, including:

[0006] Call the preset domain-brand mapping library to determine whether the domain information in the mounting link of the commercial video can be directly matched with the brand. If so, output the preliminary brand identifier; otherwise, obtain the metadata of the page pointed to by the mounting link and extract brand-related clues as brand information to be verified.

[0007] When the domain information in the embedded link of the commercial video cannot be directly matched with the brand, the corresponding content extraction strategy is executed based on the commercial type, and multi-dimensional weights are combined to obtain targeted multimodal features.

[0008] Based on the initial brand identity or targeted multimodal features, the brand information to be verified, and the brand vector library, the core brand is determined, and the core brand is standardized to obtain a standardized brand identity.

[0009] Based on standardized brand identity, combined with the characteristics of the platform to which the commercial video belongs and the target user profile data, the system calls the advertising strategy library to generate a targeted advertising strategy that is suitable for the brand type, and pushes the commercial video to the target user group according to the targeted advertising strategy. At the same time, the system monitors the advertising performance indicators in real time to obtain monitoring data, adjusts the current multi-dimensional weights based on the monitoring data, and generates a performance report.

[0010] Optionally, a preset domain-brand mapping library is invoked to determine whether the domain information in the embedded link of the commercial video can be directly matched with the brand. If so, a preliminary brand identifier is output; otherwise, the metadata of the page pointed to by the embedded link is obtained and brand-related clues are extracted as brand information to be verified, including:

[0011] Obtain the link to the commercial video and extract the domain information from the link;

[0012] The system calls a pre-defined domain-brand mapping library to determine whether the domain information in the product video's embedding link can directly match the brand.

[0013] If a direct match is possible, output the initial brand identifier;

[0014] If a direct match is not possible, obtain the HTML metadata of the linked page through a headless browser, extract brand-related clues from the metadata, and use them as brand information to be verified.

[0015] Optionally, a corresponding content extraction strategy is executed based on the order type, and targeted multimodal features are obtained by combining multidimensional weights, including:

[0016] If the order type is only video order, then adaptive frequency video frame extraction, audio extraction, OCR recognition and removal of colloquial ambiguity are performed to obtain image features, audio features and text features, and combined with multi-dimensional weights to obtain directional multimodal features.

[0017] If the order type is video order or image / text order, then adaptive frequency video frame extraction, audio extraction, OCR recognition and removal of colloquial ambiguity are performed to obtain image features, audio features and text features. At the same time, image / text titles, body descriptions, cover images, pinned comments and removal of colloquial ambiguity are performed to obtain image / text features and interactive comment features, and then combined with multidimensional weights to obtain directional multimodal features.

[0018] Optionally, the process of performing adaptive frequency video frame extraction to obtain image features includes:

[0019] Extract industry keywords from the brand information to be verified, and use the industry frame-sampling benchmark frequency table based on the industry keywords to determine the initial frame-sampling frequency.

[0020] Collect the first n seconds of the video of the commercial order, and calculate the edge pixel ratio of each frame to generate an edge pixel ratio sequence;

[0021] Calculate the first derivative sequence of the edge pixel proportion sequence, and count the total number of elements in the first derivative sequence of the edge pixel proportion sequence that are greater than the preset first derivative threshold.

[0022] The image detail features of the commercial video are determined based on the total number of elements in the first derivative sequence of the edge pixel proportion sequence that are greater than the preset first derivative threshold.

[0023] The adjustment coefficient of the initial frame sampling frequency is determined based on the image detail features, and the initial frame sampling frequency is adjusted based on the adjustment coefficient to obtain the current frame sampling frequency.

[0024] Based on the current frame extraction frequency, perform video frame extraction on the commercial video to obtain multiple extracted frame images;

[0025] For each frame-by-frame image, extract the proportion of the logo area, the proportion of the brand-specific color, and the matching degree between the scene and the brand. Based on the proportion of the logo area, the proportion of the brand-specific color, and the matching degree between the scene and the brand for each frame-by-frame image, generate the feature coordinates of each frame-by-frame image.

[0026] The mean coordinates of the feature coordinates of all extracted frames are used as the image features.

[0027] Optionally, the core brand is determined based on preliminary brand identity or targeted multimodal features, brand information to be verified, and a brand vector library, including:

[0028] If a preliminary brand identity is obtained, the preliminary brand identity is vectorized and matched with the cosine similarity of each brand standard feature vector in the brand vector library. If the match is successful, the core brand is determined.

[0029] If directional multimodal features are obtained, the platform's proprietary AI semantic model is invoked. The directional multimodal features and the brand information to be verified are combined to obtain a standardized vector of semantic-level brand information. This vector is then matched with the brand vector library for similarity. If the match is successful, the core brand is determined.

[0030] Optionally, the platform's proprietary AI semantic model is invoked to obtain a standardized vector of semantic-level brand information by combining targeted multimodal features and the brand information to be verified. This vector is then matched with a brand vector library for similarity. If a match is successful, the core brand is identified, including:

[0031] The directional multimodal features are decomposed into three-dimensional coordinates of each feature dimension, and the brand information to be verified in the directional multimodal features is mapped to three-dimensional coordinates to obtain the coordinates of the brand information to be verified.

[0032] The fusion weight of each feature dimension is calculated based on the reciprocal of the Euclidean distance between the three-dimensional coordinates of each feature dimension and the coordinates of the brand information to be verified.

[0033] The fused feature coordinates are obtained by weighting and summing the three-dimensional coordinates of each feature dimension based on the fusion weights of each feature dimension.

[0034] The fused feature coordinates are input into the platform’s proprietary AI semantic model to obtain a standardized vector of semantic-level brand information.

[0035] Calculate the cosine similarity between the standardized vector of semantic-level brand information and the standard feature vector of each brand in the brand vector library. Then, gather all brands in the brand vector library whose cosine similarity with the standardized vector of semantic-level brand information is not less than a preset similarity threshold to obtain a candidate brand set. At the same time, determine that the match is successful.

[0036] The evaluation coordinates for each candidate brand are constructed based on its historical identification accuracy, campaign conversion rate, and industry matching degree.

[0037] From the candidate brand set, the candidate brand with the smallest Manhattan distance between the corresponding evaluation coordinates and the fusion feature coordinates is selected as the core brand.

[0038] Optionally, it also includes:

[0039] If the current commercial video is set in a multi-brand scenario, then a three-level conflict resolution strategy will be initiated to identify the core brand.

[0040] Optionally, it also includes:

[0041] If the match fails, the corresponding preliminary brand identifier or semantic brand information will be stored in the new brand review pool until manual review is received. Then, the corresponding preliminary brand identifier or semantic brand information will be updated in the brand vector library and domain-brand mapping library.

[0042] Optionally, based on standardized brand identity, and combined with the characteristics of the platform to which the commercial video belongs and target user profile data, a targeted delivery strategy adapted to the brand type is generated by calling the delivery strategy library, and the commercial video is pushed to the target user group according to the targeted delivery strategy, including:

[0043] The standardized brand logo is assigned a brand type value according to the brand type assignment rules, the platform traffic conversion capability is assigned a value based on the characteristics of the platform to which the commercial video belongs, and the matching degree between the user and the core brand is assigned a value based on the target user profile data.

[0044] Based on brand type assignment, platform traffic conversion capability assignment, and user-core brand matching assignment, a coordinate system for the ad placement strategy is constructed. The benchmark strategy with the smallest Euclidean distance between the corresponding benchmark strategy coordinate and the corresponding ad placement strategy coordinate is selected from the ad placement strategy library and used as the target strategy. The target strategy is then fine-tuned based on the Euclidean distance to obtain a targeted ad placement strategy that is suitable for the brand type.

[0045] Optionally, real-time monitoring of campaign performance metrics is used to obtain monitoring data. Based on this data, the current multi-dimensional weights are adjusted, and a performance summary is generated, including:

[0046] Multiple campaign performance indicators are collected as monitoring data based on a preset time period, and the sequence records of each campaign performance indicator in the monitoring data are normalized to obtain multidimensional normalized campaign indicator sequence data.

[0047] Dimensionality reduction processing is performed on the multidimensional normalized delivery indicator sequence data to obtain three-dimensional normalized delivery indicator sequence data;

[0048] The three-dimensional effect coordinate difference between adjacent time points is calculated based on the three-dimensional normalized delivery index sequence data, and a coordinate difference sequence is generated. The first derivative of the coordinate difference sequence is then calculated to obtain the effect change rate curve.

[0049] Based on the effect change rate curve, identify all effect warning time points within the current collection period;

[0050] Map the differences in the three-dimensional effect coordinates of all effect warning time points with the differences in the brand identification feature weights to a two-dimensional optimization space to obtain multiple two-dimensional data points;

[0051] A quadratic polynomial is used to fit all data points in the two-dimensional optimization space to obtain a fitting curve. The second derivative of the fitting curve is used to determine the reasons for the fluctuation of the campaign effect. Based on the reasons for the fluctuation of the campaign effect and the corresponding second derivative, the current multidimensional weights are adjusted, and a performance report is generated.

[0052] The beneficial effects of this invention compared to existing technologies are as follows: This invention provides a method for content extraction and delivery based on multimodal commercial videos. The multimodal data encompasses various forms such as video footage, audio, cover images, and text, providing rich clues for a comprehensive understanding of the commercial video content. Through effective extraction and analysis of this multimodal data, brands can be identified more accurately, avoiding the limitations of single-modal analysis. Simultaneously, precise delivery by combining target user profiles and platform characteristics not only improves ad reach and conversion rates but also saves marketing costs for brands and enhances marketing effectiveness.

[0053] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0054] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0055] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0056] Figure 1 This is a flowchart of a content extraction and delivery method based on multimodal commercial video in an embodiment of the present invention;

[0057] Figure 2 This is a flowchart of another content extraction and delivery method based on multimodal commercial video in an embodiment of the present invention;

[0058] Figure 3 This is a flowchart illustrating the core brand identification process in an embodiment of the present invention;

[0059] Figure 4 This is a flowchart illustrating the content extraction process for commercial order videos in an embodiment of the present invention. Detailed Implementation

[0060] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0061] like Figure 1 , Figure 2 , Figure 3 As shown, this invention provides an embodiment of a method for content extraction and delivery based on multimodal commercial video, including:

[0062] Call the preset domain-brand mapping library to determine whether the domain information in the mounting link of the commercial video can be directly matched with the brand. If so, output the preliminary brand identifier; otherwise, obtain the metadata of the page pointed to by the mounting link and extract brand-related clues as brand information to be verified.

[0063] When the domain information in the embedded link of the commercial video cannot be directly matched with the brand, the corresponding content extraction strategy is executed based on the commercial type, and multi-dimensional weights are combined to obtain targeted multimodal features.

[0064] Based on the initial brand identity or targeted multimodal features, the brand information to be verified, and the brand vector library, the core brand is determined, and the core brand is standardized to obtain a standardized brand identity.

[0065] Based on standardized brand identity, combined with the characteristics of the platform to which the commercial video belongs and the target user profile data, the system calls the advertising strategy library to generate a targeted advertising strategy that is suitable for the brand type, and pushes the commercial video to the target user group according to the targeted advertising strategy. At the same time, the system monitors the advertising performance indicators in real time to obtain monitoring data, adjusts the current multi-dimensional weights based on the monitoring data, and generates a performance report.

[0066] In this embodiment, a pre-defined domain-brand mapping database is used: this is a pre-set database that stores numerous correspondences between domain names and brands, such as "lp.pinduoduo.com" corresponding to the brand "Pinduoduo". When processing commercial video orders, querying this database can quickly determine whether the domain name in the linked content directly corresponds to a specific brand.

[0067] In this embodiment, "commercial order video" refers to a video with a commercial order element, i.e., an advertising video. Brands use these videos to promote their products or services on short video or social media platforms. The content of these videos revolves around the brand and may include brand information, product demonstrations, etc., and is the subject of this embodiment.

[0068] In this embodiment, the attached link is a link included in the commercial video. Clicking the link will redirect to the relevant page, which may be the brand's official website, product purchase page, etc. It is an important way to obtain brand information.

[0069] In this embodiment, the domain name information in the mounting link is the URL portion of the mounting link, such as "lp.pinduoduo.com".

[0070] In this embodiment, the determination of whether the domain name information in the embedded link of the commercial video can directly match the brand is as follows: The domain name information extracted from the embedded link is searched and compared in a preset domain name-brand mapping database. If a completely matching domain name can be found in the database, it is considered that the brand can be directly matched; if not, it is determined that it cannot be directly matched. For example, if the domain name is "lp.pinduoduo.com", and the corresponding "Pinduoduo" can be found in the mapping database, it is considered that it can be directly matched.

[0071] In this embodiment, a preliminary brand identifier is output: when it is determined that the domain name information in the mounted link can directly match the brand, the corresponding brand identifier is obtained from the preset domain name-brand mapping library. This identifier is the preliminary brand identifier.

[0072] In this embodiment, the metadata of the page pointed to by the mounted link is obtained: when the domain information in the mounted link cannot directly match the brand, the page pointed to by the link is accessed through a headless browser, and the metadata of the page is extracted. The metadata includes various descriptive information of the page, such as title, keywords, description, etc., from which brand-related clues may be extracted to help identify the brand.

[0073] In this embodiment, brand-related clues are information related to the brand extracted from the metadata of the page pointed to by the mounted link. These may include the brand name, brand abbreviation, brand-related product names, etc., and are used to further identify the brand of the video promotion, thus compensating for information acquisition when the domain name cannot be directly matched with the brand.

[0074] In this embodiment, multi-dimensional weighting is used: when processing the multimodal features of commercial videos, different weight values ​​are assigned to features of different modalities. For example, for visual features, audio features, and text features, different weights are set according to their importance to brand identification. The initial assignment rule is: visual features 0.4, audio features 0.3, and text features 0.3, which can be dynamically adjusted based on the campaign performance.

[0075] In this embodiment, the targeted multimodal features are obtained by executing corresponding content extraction strategies based on the type of the order, such as adaptive frequency video frame extraction, audio extraction, OCR recognition and removal of colloquial ambiguity for video orders, and extraction of image and text titles, body descriptions and other multimodal features for image and text orders. These features are targeted at specific video content of the order.

[0076] In this embodiment, the brand vector library is a database that stores a large number of standard feature vectors of brands, containing a large amount of vector information about brands.

[0077] In this embodiment, the core brand is the brand that is actually promoted in the commercial video, which is finally determined after matching the preliminary brand identity or targeted multimodal features, the brand information to be verified, and the brand vector library.

[0078] In this embodiment, the core brand is standardized to obtain a standardized brand identity: the determined core brand is standardized according to certain rules, such as mapping the "Li Auto L9" model to "Li Auto" to obtain a standard and unified brand identity, which facilitates subsequent operations such as formulating marketing strategies based on the identity, and also makes the brand output more standardized and unified.

[0079] In this embodiment, the characteristics of the platform to which the commercial video belongs are as follows: different short video or social media platforms have their own characteristics, such as user groups, traffic distribution, and promotion rules. For example, Xiaohongshu platform users are mostly young women, and beauty and fashion content is more popular; Douyin platform has a wide user base, and various types of content have high popularity.

[0080] In this embodiment, target user profile data refers to the set of characteristics of target users depicted by analyzing and integrating various user data, such as age, gender, interests, and consumption habits. For example, the target user profile of a certain brand might be males aged 25-35 who enjoy outdoor sports and have a certain level of purchasing power.

[0081] In this embodiment, the campaign strategy library is a database that pre-stores various campaign strategies. These strategies are formulated based on different brand types, platform characteristics, and target user profiles. For example, a campaign strategy for a beauty brand targeting young female users on the Xiaohongshu platform might focus on aesthetically pleasing images and videos, influencer recommendations, and other methods.

[0082] In this embodiment, the targeted advertising strategy adapted to the brand type is as follows: This strategy, specifically designed for precise targeting of the brand to a particular user group on a specific platform, is obtained by combining the brand type determined by standardized brand identity, the characteristics of the platform to which the commercial video belongs, and target user profile data, and is selected and fine-tuned from the advertising strategy library. For example, based on the characteristics of a beauty brand and the preferences of Xiaohongshu platform users, a strategy suitable for promoting the beauty brand on the Xiaohongshu platform is formulated, including the advertising time, advertising format, and advertising content.

[0083] In this embodiment, promotional videos are pushed to the target user group according to a targeted delivery strategy: based on a pre-defined targeted delivery strategy that matches the brand type, promotional videos are pushed to user groups that match the target user profile. For example, according to a delivery strategy tailored for a sports brand, promotional videos for that brand are pushed to users who like sports on the Douyin platform.

[0084] In this embodiment, monitoring data is obtained by real-time monitoring of campaign performance indicators: during the campaign for commercial videos, various indicator data that can reflect the campaign performance are continuously tracked and collected, such as click-through rate, conversion rate, number of views, and user dwell time.

[0085] In this embodiment, the performance summary is a report generated after analyzing and processing monitoring data obtained from real-time monitoring of campaign performance indicators. The report includes a summary of campaign performance, an analysis of the reasons for fluctuations in performance, and suggestions for adjusting multi-dimensional weights based on the analysis results.

[0086] To accurately identify brand cues, a pre-defined domain-brand mapping library is invoked to determine whether the domain information in the embedded link of the commercial video can directly match the brand. If so, a preliminary brand identifier is output; otherwise, the metadata of the page pointed to by the embedded link is obtained, and brand-related cues are extracted as brand information to be verified, including:

[0087] Obtain the link to the commercial video and extract the domain information from the link;

[0088] The system calls a pre-defined domain-brand mapping library to determine whether the domain information in the product video's embedding link can directly match the brand.

[0089] If a direct match is possible, output the initial brand identifier;

[0090] If a direct match is not possible, obtain the HTML metadata of the linked page through a headless browser, extract brand-related clues from the metadata, and use them as brand information to be verified.

[0091] In this embodiment, a headless browser is used to obtain the HTML metadata of the linked page, and brand-related clues are extracted from the metadata. A headless browser is a browser without a graphical interface that can simulate browser operations in the background to access web pages. After accessing the page linked by the headless browser, the HTML metadata of that page can be obtained.

[0092] HTML metadata contains various descriptive information about a webpage, such as the page title, search engine keywords, and page description. For example, the title of a brand's product introduction page might directly include the brand name, such as "[Brand Name] Smartwatch, Ushering in a New Era of Smart Living," and keywords might include "[Brand Name] Watch, Smart Wearables," etc. Extracting brand-related information from this metadata, such as the brand name, brand abbreviation, and names of related products, provides brand-related clues.

[0093] like Figure 3 As shown, to deeply mine the characteristics of commercial order video content, a content extraction strategy based on commercial order type is proposed, and multi-dimensional weights are combined to obtain targeted multimodal features, including:

[0094] If the order type is only video order, then adaptive frequency video frame extraction, audio extraction, OCR recognition and removal of colloquial ambiguity are performed to obtain image features, audio features and text features, and combined with multi-dimensional weights to obtain directional multimodal features.

[0095] If the order type is video order or image / text order, then adaptive frequency video frame extraction, audio extraction, OCR recognition and removal of colloquial ambiguity are performed to obtain image features, audio features and text features. At the same time, image / text titles, body descriptions, cover images, pinned comments and removal of colloquial ambiguity are performed to obtain image / text features and interactive comment features, and then combined with multidimensional weights to obtain directional multimodal features.

[0096] In this embodiment, adaptive frequency video frame extraction, audio extraction, OCR recognition, and removal of colloquial ambiguity are performed to obtain image features, audio features, and text features, which are then combined with multidimensional weights to obtain directional multimodal features.

[0097] Audio extraction: Isolate the audio part from the commercial video, and analyze the audio, such as identifying key audio information such as brand names and advertising slogans mentioned in it. These information constitute audio features.

[0098] OCR recognition and removal of colloquial ambiguity: Use OCR technology to recognize the text content in the video frame, and at the same time remove the ambiguity brought by colloquial expressions. For example, convert "pxx" to "Pinduoduo" to obtain accurate text information as text features.

[0099] Obtain directional multi-modal features by combining multiple weights: Assign different weights to the frame features, audio features, and text features respectively. For example, the weight of the frame feature is 0.4, the weight of the audio feature is 0.3, and the weight of the text feature is 0.3. These features are fused together through weighted calculation to obtain directional multi-modal features, comprehensively reflecting the video content to assist brand recognition.

[0100] In this embodiment, the commercial order types are video commercial orders and graphic commercial orders: In this scenario, the commercial order contains both video-form content and graphic-form content. For the video part, perform the same adaptive frequency video frame extraction, audio extraction, OCR recognition and removal of colloquial ambiguity operations as the above-mentioned pure video commercial order to obtain frame features, audio features and text features. For the graphic part:

[0101] Extract graphic titles, text descriptions, cover images, top comments and remove colloquial ambiguity: Extract titles and text descriptions from graphic commercial orders, analyze the cover images to extract key information, obtain top comments, and remove the colloquial ambiguity in them. For example, the title is "Come and see this awesome [brand name] good", the text description may detail the product features, the cover image may show the brand logo or product appearance, and the top comment may be the user's evaluation of the product. After processing these information, they are respectively formed into graphic features and interactive comment features.

[0102] Obtain directional multi-modal features by combining multiple weights: Assign weights to the frame, audio, and text features obtained from the video part, and the graphic features and interactive comment features of the graphic part according to their respective importance levels. For example, the weight of the frame feature is 0.3, the weight of the audio feature is 0.2, the weight of the text feature is 0.2, the weight of the graphic feature is 0.2, and the weight of the interactive comment feature is 0.1. These features are fused through weighted calculation to obtain directional multi-modal features that more comprehensively and accurately reflect the content of the commercial order.

[0103] To achieve adaptive and accurate extraction of video frame features, a process of performing adaptive frequency video frame extraction to obtain frame features is proposed, including:

[0104] Extract industry keywords from the brand information to be verified, and use the industry frame-sampling benchmark frequency table based on the industry keywords to determine the initial frame-sampling frequency.

[0105] Collect the first n (assuming n=5) seconds of the video of the commercial order, and calculate the edge pixel ratio of each frame to generate an edge pixel ratio sequence;

[0106] Calculate the first derivative sequence of the edge pixel proportion sequence, and count the total number of elements in the first derivative sequence of the edge pixel proportion sequence that are greater than the preset first derivative threshold.

[0107] The image detail features of the commercial video are determined based on the total number of elements in the first derivative sequence of the edge pixel proportion sequence that are greater than the preset first derivative threshold.

[0108] The adjustment coefficient of the initial frame sampling frequency is determined based on the image detail features, and the initial frame sampling frequency is adjusted based on the adjustment coefficient to obtain the current frame sampling frequency.

[0109] Based on the current frame extraction frequency, perform video frame extraction on the commercial video to obtain multiple extracted frame images;

[0110] For each extracted frame image, extract the logo area ratio (target brand logo area / total image area), brand-specific color ratio, and scene-brand matching degree (0-1 score, e.g., 0.9 for a technology brand corresponding to a laboratory scene). Based on the logo area ratio (number of brand standard color pixels / total number of pixels), brand-specific color ratio (the proportion of brand-specific color pixels to the total number of pixels in the image, e.g., if a brand's specific color is blue, then blue pixels account for 30% of the total pixels in this extracted frame image), and scene-brand matching degree (based on some preset rules or algorithms to determine whether the scene in the image matches the scene where the brand usually appears, represented by a value between 0 and 1, e.g., a matching degree of 0.8 means a high matching degree between the scene and the brand), generate feature coordinates for each extracted frame image. For example, use the logo area ratio as the X-axis value, the brand-specific color ratio as the Y-axis value, and the scene-brand matching degree as the Z-axis value to form feature coordinates like (0.2, 0.3, 0.8).

[0111] The mean coordinates of the feature coordinates of all extracted frames are used as the image features.

[0112] In this embodiment, industry keywords are extracted from the brand information to be verified: the brand information includes brand-related content obtained from channels such as metadata of pages linked to by embedded links. From this information, words that can represent the industry are identified. For example, if the brand information involves "smartphone reviews," then "smartphone" and "review" are likely industry keywords.

[0113] In this embodiment, the initial frame-sampling frequency is determined by calling the industry frame-sampling reference frequency table based on industry keywords. This table stores reference values ​​for frame-sampling frequencies corresponding to different industries. Once an industry keyword is obtained, it is used as an index to look up the corresponding frame-sampling frequency in the table; this frequency is the initial frame-sampling frequency. For example, if the industry keyword is "food preparation," the corresponding initial frame-sampling frequency in the industry frame-sampling reference frequency table might be 1 frame per second.

[0114] In this embodiment, the industry frame-slicing reference frequency table is a pre-established table that records the correspondence between various industries and frame-slicing frequencies, storing 20+ industry reference values, such as beauty (2 frames / second), automobiles (3 frames / second), and games (5 frames / second).

[0115] In this embodiment, the edge pixel percentage of each frame is calculated to generate an edge pixel percentage sequence: Each frame of the commercial video is processed, the number of edge pixels in the frame is counted, and then divided by the total number of pixels to obtain the edge pixel percentage of that frame. The edge pixel percentage of all frames is calculated sequentially and arranged in the order of the frames to form a sequence. For example, the edge pixel percentage of the first frame of a video is 0.2, the second is 0.25, and so on, forming the sequence [0.2, 0.25, ...].

[0116] In this embodiment, the first derivative sequence of the edge pixel proportion sequence is calculated: the first derivative of the edge pixel proportion sequence is calculated to obtain a new sequence. For each element in the sequence, the difference between it and its neighboring elements is calculated. For example, for the sequence [0.2, 0.25, 0.3], the first derivative sequence is [0.05, 0.05]. The first derivative reflects the rate of change of the edge pixel proportion, which helps to analyze the speed of changes in image details.

[0117] In this embodiment, a preset first-derivative threshold is used: a fixed value set based on experience and experiments, which is used to compare with elements in the first-derivative sequence of the edge pixel proportion sequence. For example, the preset first-derivative threshold is set to 0.03, which is the standard for judging whether the degree of change in image detail is significant.

[0118] In this embodiment, the total number of elements in the first derivative sequence of the edge pixel proportion sequence that are greater than a preset first derivative threshold is counted: each element in the first derivative sequence of the edge pixel proportion sequence is compared with the preset first derivative threshold, and the number of elements greater than the threshold is counted. For example, if the first derivative sequence is [0.05, 0.02, 0.04, 0.01], and the preset first derivative threshold is 0.03, then there are 2 elements greater than the threshold. This number can reflect the frequency of changes in image detail.

[0119] In this embodiment, the image detail features of the commercial video are determined based on the total number of elements in the first derivative sequence of the edge pixel proportion sequence that are greater than a preset first derivative threshold. If the total number of elements is large, it indicates that the image details change frequently and are rich in detail; if the number is small, it indicates that the image details change relatively slowly and are relatively scarce. For example, if the total number of elements greater than the preset first derivative threshold is greater than 10, the image details are considered rich; if it is less than 5, the image details are considered relatively simple. This is used to determine the image detail features.

[0120] In this embodiment, the adjustment coefficient for the initial frame sampling frequency is determined based on the image detail features: the adjustment coefficient is determined according to the image detail feature judgment result. If the image detail is rich, the adjustment coefficient may be increased to better capture image information, such as setting it to 1.5; if the image detail is simple, the adjustment coefficient may be decreased, such as setting it to 0.8. The adjustment coefficient is used to adjust the initial frame sampling frequency to adapt to the actual image conditions.

[0121] In this embodiment, the initial frame-sampling frequency is adjusted based on an adjustment coefficient to obtain the current frame-sampling frequency: the initial frame-sampling frequency is multiplied by the adjustment coefficient to obtain the current frame-sampling frequency. For example, if the initial frame-sampling frequency is 2 frames per second and the adjustment coefficient is 1.5, then the current frame-sampling frequency is 2 × 1.5 = 3 frames per second.

[0122] In this embodiment, video frame extraction is performed on the commercial video based on the current frame extraction frequency to obtain multiple extracted frame images: the commercial video is subjected to frame extraction operation according to the calculated current frame extraction frequency. For example, if the current frame extraction frequency is 3 frames per second, then 3 frames are extracted from the video per second, and multiple extracted frame images can be obtained after continuous extraction.

[0123] like Figure 2 As shown, to accurately determine the core brand, a method is proposed that uses preliminary brand identifiers or targeted multimodal features, brand information to be verified, and a brand vector library to determine the core brand, including:

[0124] If a preliminary brand identity is obtained, the preliminary brand identity is vectorized and matched with the cosine similarity of each brand standard feature vector in the brand vector library. If the match is successful, the core brand is determined.

[0125] If directional multimodal features are obtained, the platform's proprietary AI semantic model is invoked. The directional multimodal features and the brand information to be verified are combined to obtain a standardized vector of semantic-level brand information. This vector is then matched with the brand vector library for similarity. If the match is successful, the core brand is determined.

[0126] In this embodiment, the initial brand identifier is vectorized and matched with the cosine similarity of each brand standard feature vector in the brand vector library. If the match is successful, the core brand is determined.

[0127] Vectorization transforms the initial brand identity into a vector form, enabling computers to process and compare it mathematically. For example, a brand name can be converted into a multi-dimensional vector using a specific algorithm, where each dimension represents a certain characteristic of the brand, such as a quantitative description of brand style, product type, etc.

[0128] The initial brand identity vector library stores standard feature vectors for numerous brands. These vectors are generated according to certain rules and are used to represent the characteristics of each brand.

[0129] Cosine similarity matching calculates the cosine similarity between the vector converted from the initial brand identity and the standard feature vector of each brand in the vector library. The cosine similarity value ranges from -1 to 1. The closer it is to 1, the smaller the angle between the two vectors, which means that the two brand features are more similar.

[0130] If the calculated cosine similarity exceeds a preset threshold (e.g., 0.8), the match is considered successful, and the brand in the corresponding vector library is the core brand. For example, the initial brand identifier "Huawei Mobile" has a cosine similarity of 0.9 with the standard feature vector of the "Huawei" brand in the vector library after vectorization, which exceeds the threshold, thus "Huawei" is determined to be the core brand.

[0131] In this embodiment, the standardized vector of semantic-level brand information is as follows: After processing the targeted multimodal features and the brand information to be verified, the platform's proprietary AI semantic model transforms them into a vector with a unified format and fixed dimensions. This is the standardized vector of semantic-level brand information. For example, the model extracts the core semantic features of the brand, such as brand positioning and main product characteristics, from multimodal information such as video footage, audio, and text, as well as brand cues to be verified. These features are then quantified into numerical values ​​for each dimension of the vector, forming a standardized vector. In this way, different brand information is presented in the same standardized vector form, which facilitates similarity matching with vectors in the brand vector library.

[0132] In this embodiment, a similarity match is performed with a brand vector library. If the match is successful, the core brand is determined.

[0133] The standardized vector of semantic brand information is compared with the standard feature vector of each brand in the brand vector library to calculate the similarity. The calculation method is similar to the cosine similarity calculation mentioned earlier.

[0134] Similarly, a similarity threshold is set. If the calculated similarity is greater than the threshold, a successful match is determined, and the corresponding brand in the brand vector library becomes the core brand. For example, the standardized vector of semantic-level brand information and the standard feature vector of the "Xiaomi" brand in the brand vector library have a similarity of 0.85, exceeding the threshold, thus "Xiaomi" is identified as the core brand. In this way, by utilizing the rich brand feature vectors in the brand vector library, the brand that best matches the semantic information of the current commercial video is accurately identified and determined as the core brand.

[0135] To accurately identify the core brand, a method is proposed that the platform's proprietary AI semantic model be used to obtain a standardized vector of semantic-level brand information by combining targeted multimodal features and the brand information to be verified. This vector is then matched with a brand vector library for similarity. If a match is successful, the core brand is identified, including:

[0136] The directional multimodal features are decomposed into three-dimensional coordinates of each feature dimension, and the brand information to be verified in the directional multimodal features is mapped to three-dimensional coordinates to obtain the coordinates of the brand information to be verified.

[0137] The fusion weight of each feature dimension is calculated based on the reciprocal of the Euclidean distance between the three-dimensional coordinates of each feature dimension and the coordinates of the brand information to be verified.

[0138] The fused feature coordinates are obtained by weighting and summing the three-dimensional coordinates of each feature dimension based on the fusion weights of each feature dimension.

[0139] The fused feature coordinates are input into the platform’s proprietary AI semantic model to obtain a standardized vector of semantic-level brand information.

[0140] Calculate the cosine similarity between the standardized vector of semantic-level brand information and the standard feature vector of each brand in the brand vector library. Then, gather all brands in the brand vector library whose cosine similarity with the standardized vector of semantic-level brand information is not less than a preset similarity threshold to obtain a candidate brand set. At the same time, determine that the match is successful.

[0141] The evaluation coordinates for each candidate brand are constructed based on its historical identification accuracy, campaign conversion rate, and industry matching degree.

[0142] From the candidate brand set, the candidate brand with the smallest Manhattan distance between the corresponding evaluation coordinates and the fusion feature coordinates is selected as the core brand.

[0143] In this embodiment, the directional multimodal features are decomposed into three-dimensional coordinates for each feature dimension: directional multimodal features include various features extracted from multimodal information in the commercial video (such as video images, audio, text, etc.). For ease of analysis and processing, these features are represented using three-dimensional coordinates, with each dimension corresponding to one aspect of the feature. For example, the directional multimodal feature vector is decomposed into text features, image features, and audio features:

[0144] Textual features: For the text after removing colloquial ambiguity, the keyword weights are calculated using the TF-IDF algorithm. The weights of the top 3 keywords are selected as X, Y, and Z to construct the three-dimensional coordinates of the textual features T=(X,Y,Z) (weight values ​​are normalized to the 0-1 range).

[0145] Image features: The logo clarity, scene matching degree, and color features identified from video frame extraction are each assigned a value of X. Y Z Construct the three-dimensional coordinates of the image features I=(X ,Y Z (The value ranges from 0 to 1, with higher resolution resulting in larger values).

[0146] Audio features: The frequency of brand phrases mentioned in the audio, the matching degree of the exclusive background music, and the degree of background noise interference are each assigned a value of X. Y Z Construct the three-dimensional coordinates of the audio features A=(X ,Y Z (The more times it is mentioned, the higher the match and the lower the interference, the larger the value, normalized to the 0-1 range).

[0147] In this embodiment, the brand information to be verified in the directional multimodal features is mapped to three-dimensional coordinates to obtain the coordinates of the brand information to be verified. In order to process this information in a unified manner with other feature dimensions, specific mapping rules are used to transform this information into three-dimensional coordinates. For example, keyword weights, scene relevance, and spoken mentions are extracted from the brand information to be verified to construct the coordinates, with dimensions 0-1.

[0148] In this embodiment, the fusion weight of each feature dimension is calculated based on the reciprocal of the Euclidean distance between the three-dimensional coordinates of each feature dimension and the coordinates of the brand information to be verified:

[0149] The closer the Euclidean distance, the higher the relevance between the feature dimension and the brand information to be verified. The ratio of the reciprocal of the Euclidean distance between the three-dimensional coordinates of each feature dimension and the coordinates of the brand information to be verified to the sum of the reciprocals of the Euclidean distances between the three-dimensional coordinates of all feature dimensions and the coordinates of the brand information to be verified is used as the weight of the corresponding feature dimension. In this way, the closer the feature dimension is, the greater its weight, and the greater its influence on the result in the subsequent fusion process.

[0150] In this embodiment, the three-dimensional coordinates of each feature dimension are weighted and summed based on the fusion weights of each feature dimension to obtain the fused feature coordinates:

[0151] Multiply the 3D coordinates of each feature dimension by its corresponding fusion weight, and then sum the products to obtain the fused feature coordinates.

[0152] In this embodiment, the platform-specific AI semantic model is a BERT-based fine-tuned model built to solve the problem of core brand identification in multi-brand coexistence scenarios. It is designed specifically for the language and content characteristics of the platform, aiming to accurately determine the core brand promoted in a commercial video through multimodal information analysis.

[0153] The training process inputs a large amount of commercial video data from a specific platform. This data encompasses multimodal information such as video footage, audio, and text, along with corresponding accurate brand labeling information, resulting in 128-dimensional fused feature coordinates. After preprocessing, the multimodal information is transformed into processable feature vectors. For example, the directional multimodal features are decomposed into text, video, and audio feature dimensions, and three-dimensional coordinates are constructed for each. Simultaneously, the brand information to be verified is also mapped to three-dimensional coordinates. The fusion weights are obtained by calculating the reciprocal of the Euclidean distance between each feature dimension and the coordinates of the brand information to be verified, and then a weighted sum is performed to obtain the fused feature coordinates. The model's output is a 32-dimensional standardized brand vector containing semantic-level brand information. Through multi-layered neural network operations, the model performs deep semantic analysis and processing on the fused feature coordinates, extracting semantic features and transforming them into vectors of fixed dimensions and a unified format.

[0154] In this embodiment, the fused feature coordinates are input into the platform's proprietary AI semantic model to obtain a standardized vector of semantic-level brand information:

[0155] The platform-specific AI semantic model is designed to address the language and content characteristics of the platform. Using fused feature coordinates as input, the model performs deep semantic analysis and processing, transforming them into a standardized vector of semantic-level brand information. The model may extract semantic features from the fused feature coordinates through multi-layered neural network operations and convert them into a fixed-dimensional, uniformly formatted vector.

[0156] In this embodiment, the standard feature vector of a brand is as follows: Each brand in the brand vector library has a corresponding standard feature vector. These vectors are obtained by analyzing and quantifying various information about the brand (such as brand positioning, product characteristics, market image, etc.). For example, the brand's core values, target audience characteristics, and main product types are represented by different dimensions of the vector, forming a multi-dimensional vector to represent the brand's characteristics.

[0157] In this embodiment, a preset similarity threshold is used: during similarity matching, a pre-set value is used to determine whether the similarity between two vectors (the standardized vector of semantic-level brand information and the standard feature vector of the brand) is high enough to determine a successful match. For example, the preset similarity threshold is set to 0.7.

[0158] In this example, the historical identification accuracy, campaign conversion rate, and industry matching degree for each brand are as follows:

[0159] Historical recognition accuracy refers to the percentage of times a brand is correctly recognized out of the total number of recognitions in the past. For example, if the "Apple" brand is recognized 100 times and correctly 95 times, its historical recognition accuracy is 95%. A high historical recognition accuracy indicates that the brand has performed well in past recognitions and is more likely to be the core brand in the current commercial video.

[0160] Conversion rate refers to the ratio of the number of users who convert (e.g., purchase a product, register as a member) after a brand's advertisement is launched to the number of ad impressions or clicks. For example, if a brand's advertisement is viewed 1000 times and 50 users complete a purchase, its conversion rate is 5%. The conversion rate reflects the actual effect of a brand's advertising. The higher the conversion rate, the better the brand's performance in marketing, and the greater its likelihood of becoming a core brand.

[0161] Industry fit is used to measure the degree to which a brand aligns with the industry covered in the commercial video. For example, if the commercial video is about electronic product reviews, the "Huawei" brand has a high fit in the electronic product industry, while the "Coca-Cola" brand has a lower fit. By assessing industry fit, the relevance of a brand to the content of the current commercial video can be determined, helping to identify the core brand.

[0162] In this embodiment, an evaluation coordinate system is constructed for each candidate brand based on its historical recognition accuracy, campaign conversion rate, and industry matching degree. The historical recognition accuracy, campaign conversion rate, and industry matching degree of each candidate brand are used as one dimension of a three-dimensional coordinate system to construct the evaluation coordinate system. For example, if a candidate brand has a historical recognition accuracy of 90%, a campaign conversion rate of 8%, and an industry matching degree of 0.9, its evaluation coordinate system would be (0.9, 0.08, 0.9).

[0163] In this embodiment, Manhattan distance: Manhattan distance measures the distance between two points in a Cartesian coordinate system. Unlike Euclidean distance, which calculates the straight-line distance between two points, it calculates the sum of distances traveled along the coordinate axes. In this embodiment, the difference between the candidate brand and the brand characteristics reflected by the multimodal features of the commercial video is measured by calculating the Manhattan distance between the evaluation coordinates and the fused feature coordinates of the candidate brand. The smaller the distance, the closer the candidate brand is to the brand characteristics of the commercial video, and the more likely it is to be the core brand.

[0164] To effectively address the issue of determining the core brand in multi-brand scenarios, the following additional measures are proposed:

[0165] If the current commercial video is set in a multi-brand scenario, then a three-level conflict resolution strategy will be initiated to identify the core brand.

[0166] In this embodiment, a multi-brand coexistence scenario occurs when multiple brand-related information appears simultaneously within the content of a commercial video. For example, a car review video might mention the performance of BMW cars, showcase the advantages of Pirelli tires, and introduce the features of Harman Kardon audio systems—this is a multi-brand coexistence scenario. In such cases, conventional brand identification methods may struggle to determine the core brand the video is truly promoting, as the interference from multiple brand information increases the difficulty of accurate identification.

[0167] In this embodiment, a three-level conflict resolution strategy is employed. This strategy is specifically designed for multi-brand coexistence scenarios and uses three levels of judgment to determine the core brand.

[0168] Level 1: Link Resolution: Prioritize resolving links embedded in commercial videos. Because embedded links often directly link to a brand's official or promotional pages, analyzing the domain-brand mapping database or the metadata of the linked page can quickly identify brands closely related to the link. For example, if the domain of the embedded link is "bmw.com.cn", the domain-brand mapping database can directly identify the associated brand as "BMW". Therefore, at this level, "BMW" is determined to be the core brand.

[0169] Level Two: Platform Identification: If the core brand cannot be clearly identified through link analysis, proceed to this level. Analyze platform identification information appearing in the video. For example, some e-commerce platforms and automotive platforms may have unique logos, slogans, or specific cooperation models in the video. If the "Autohome" platform logo appears multiple times in the video, and the content revolves around the platform's featured services while mentioning multiple brands, then brands related to the "Autohome" platform (such as brands heavily promoted on the platform) are more likely to be the core brand.

[0170] Level 3: AI Analysis: When the first two levels fail to determine the core brand, the platform's proprietary AI semantic model is used to conduct in-depth analysis of targeted multimodal features and brand information to be verified. Through a comprehensive understanding of multimodal information such as video footage, audio, and text, and combined with a brand vector library for similarity matching, the core brand is determined from multiple brands using methods such as coordinate calculation and Manhattan distance, as described earlier. For example, through AI analysis, factors such as the frequency, duration, and semantic association strength of each brand appearing in the video are comprehensively considered to ultimately determine the core brand.

[0171] like Figure 3 As shown, to improve the brand information database and ensure the accuracy of brand information, the following additional measures are proposed:

[0172] If the match fails, the corresponding preliminary brand identifier or semantic brand information will be stored in the new brand review pool until manual review is received. Then, the corresponding preliminary brand identifier or semantic brand information will be updated in the brand vector library and domain-brand mapping library.

[0173] In this embodiment, a new brand verification pool is added: this is a place to temporarily store brand information that failed to match. When the system processes commercial video, whether it is the initial brand identifier obtained directly through domain name matching or the semantic-level brand information obtained through a series of analyses, if a similarity match with the brand vector library fails, this brand information will be stored in the new brand verification pool. It centrally manages this information, which may be from new brands or has questionable identification, pending further processing.

[0174] In this embodiment, the manual review process includes feedback: relevant staff will conduct a detailed assessment of the brand information based on certain standards and experience, such as confirming the brand's existence, the accuracy of the brand information, and its relevance to the video content. If the review is successful, feedback will be provided indicating that the manual review has been approved.

[0175] In this embodiment, the corresponding preliminary brand identifier or semantic-level brand information is updated to the brand vector library and the domain-brand mapping library: once manual review approval is received, the preliminary brand identifier or semantic-level brand information previously stored in the new brand review pool is added to the brand vector library and the domain-brand mapping library. In the brand vector library, a corresponding standard feature vector is generated for this new brand. In the domain-brand mapping library, if the brand has corresponding domain information, a corresponding mapping relationship is also added. For example, if the new brand "[New Brand Name]" has an official website domain "[New Domain].com", a mapping relationship between "[New Domain].com" and "[New Brand Name]" will be established in the domain-brand mapping library.

[0176] To achieve precise advertising video delivery strategies, this paper proposes a method based on standardized brand identity, combined with the characteristics of the platform to which the advertising video belongs and target user profile data. This method utilizes a delivery strategy library to generate targeted delivery strategies adapted to the brand type, and then pushes the advertising video to the target user group according to these strategies. This includes:

[0177] The standardized brand logo is assigned a brand type value according to the brand type assignment rules, the platform traffic conversion capability is assigned a value based on the characteristics of the platform to which the commercial video belongs, and the matching degree between the user and the core brand is assigned a value based on the target user profile data.

[0178] Based on brand type assignment, platform traffic conversion capability assignment, and user-core brand matching assignment, a coordinate system for the ad placement strategy is constructed. The benchmark strategy with the smallest Euclidean distance between the corresponding benchmark strategy coordinate and the corresponding ad placement strategy coordinate is selected from the ad placement strategy library and used as the target strategy. The target strategy is then fine-tuned based on the Euclidean distance to obtain a targeted ad placement strategy that is suitable for the brand type.

[0179] In this embodiment, the brand type assignment rules are a set of pre-defined criteria used to assign corresponding values ​​to brands based on factors such as brand attributes, characteristics, and industry category. For example, brands can be categorized by industry into technology, beauty, and food, with technology brands assigned a value of 1, beauty brands a value of 2, and food brands a value of 3. Alternatively, based on the brand's market positioning, high-end brands can be assigned a value of 5, mid-to-high-end brands a value of 4, and mass-market brands a value of 3, and so on.

[0180] In this embodiment, the standardized brand identifier is assigned a brand type value according to the brand type assignment rules: After obtaining the standardized brand identifier, the corresponding value for the brand is determined based on the brand type assignment rules. For example, if the standardized brand identifier is "Apple", according to the rules, the value assigned to a technology brand is 1. Since "Apple" belongs to the technology brand category, its brand type value is 1.

[0181] In this embodiment, the platform's traffic conversion capability is assigned a value based on the characteristics of the platform to which the commercial video belongs: different platforms have different characteristics such as traffic scale, user activity, and user conversion efficiency. Based on these platform characteristics, the platform's traffic conversion capability is evaluated and assigned a corresponding value. For example, a platform like Douyin, with a large user base, high activity, and mature commercial conversion, might be assigned a traffic conversion capability value of 8; while some relatively niche platforms with smaller traffic scale might be assigned a value of 3.

[0182] In this embodiment, the matching degree between users and the core brand is determined based on target user profile data. Target user profile data includes various characteristics of target users, such as age, gender, interests, and consumption habits. By analyzing the degree of alignment between this data and the core brand's target audience and product characteristics, the matching degree between users and the core brand is determined and assigned a corresponding value. For example, if the core brand is a beauty brand targeting young, fashionable women, and the target user profile data shows that the target users are mainly women aged 20-30 with a strong interest in fashion and beauty, the matching degree is high, and the value might be 9; if the target user profile differs significantly from the core brand's target audience, the matching degree is low, and the value might be 2.

[0183] In this embodiment, a coordinate system is constructed based on brand type assignment, platform traffic conversion capability assignment, and user-core brand matching degree assignment: a three-dimensional coordinate system is built by using brand type assignment, platform traffic conversion capability assignment, and user-core brand matching degree assignment as the three dimensions of the coordinate system. For example, if the brand type assignment is 3, the platform traffic conversion capability assignment is 6, and the user-core brand matching degree assignment is 7, then the coordinate system for the advertising strategy is (3, 6, 7).

[0184] In this embodiment, the target strategy is fine-tuned based on Euclidean distance to obtain a targeted delivery strategy suitable for the brand type: After finding the target strategy, it is adjusted appropriately according to the size of the Euclidean distance and actual business needs. For example, if the Euclidean distance is large, such as 5, it indicates that the current delivery strategy's coordinates differ significantly from the original applicable scenario of the target strategy. A substantial adjustment to the target strategy may be necessary. For instance, in terms of delivery channel selection, the target strategy originally focused on social media platforms; considering the current brand-user matching characteristics, it may be adjusted to increase delivery efforts on e-commerce platforms. Regarding delivery time, the target strategy originally concentrated on 8-10 PM; based on the current platform traffic conversion capabilities and user activity time analysis, it may be adjusted to deliver during 3-5 PM and 9-11 PM. In terms of content focus, given the differences in brand type, the original focus on product appearance display may now be adjusted to highlight product function introductions. If the Euclidean distance is small, such as 1, it indicates that the delivery strategy's coordinates are close to the original applicable scenario of the target strategy, requiring only minor adjustments. For example, in terms of channel selection, the investment in a specific niche platform can be appropriately increased for the original social media platforms. For example, for fashion brands, in addition to the original general social media platforms, investment in fashion-related vertical platforms can be increased. In terms of the timing of the campaign, the original 8-10 pm campaign time can be slightly adjusted to 7-9 pm.

[0185] To effectively monitor and optimize campaign performance, this paper proposes a method for real-time monitoring of campaign performance metrics to obtain monitoring data. Based on this data, the paper adjusts the current multi-dimensional weights and generates a performance summary, including:

[0186] Multiple campaign performance indicators are collected as monitoring data based on a preset time period, and the sequence records of each campaign performance indicator in the monitoring data are normalized to obtain multidimensional normalized campaign indicator sequence data.

[0187] Dimensionality reduction processing is performed on the multidimensional normalized delivery indicator sequence data to obtain three-dimensional normalized delivery indicator sequence data;

[0188] The three-dimensional effect coordinate difference between adjacent time points is calculated based on the three-dimensional normalized delivery index sequence data, and a coordinate difference sequence is generated. The first derivative of the coordinate difference sequence is then calculated to obtain the effect change rate curve.

[0189] Based on the effect change rate curve, identify all effect warning time points within the current collection period;

[0190] Map the differences in the three-dimensional effect coordinates of all effect warning time points with the differences in the brand identification feature weights to a two-dimensional optimization space to obtain multiple two-dimensional data points;

[0191] A quadratic polynomial is used to fit all data points in the two-dimensional optimization space to obtain a fitting curve. The second derivative of the fitting curve is used to determine the reasons for the fluctuation of the campaign effect. Based on the reasons for the fluctuation of the campaign effect and the corresponding second derivative, the current multidimensional weights are adjusted, and a performance report is generated.

[0192] In this embodiment, the preset time period refers to a fixed time period that is pre-set for periodically collecting data on the effectiveness of the campaign. For example, it can be set to daily, weekly, or monthly.

[0193] In this embodiment, multiple performance metrics are collected based on a preset time period: within the set time period, data on various metrics that reflect the effectiveness of the commercial video campaign are collected. These metrics may include click-through rate, conversion rate, number of views, user dwell time, number of comments, number of likes, etc.

[0194] In this embodiment, the sequence records of each campaign performance indicator in the monitoring data are normalized to obtain multidimensional normalized campaign performance indicator sequence data. To facilitate unified analysis and comparison, it is necessary to normalize the data for each indicator. Normalization maps the data to a specific interval, typically [0,1]. All click-through rate data are converted to the [0,1] interval. After performing this processing on each campaign performance indicator, multidimensional normalized campaign performance indicator sequence data is obtained.

[0195] In this embodiment, dimensionality reduction processing is performed on the multidimensional normalized delivery index sequence data to obtain three-dimensional normalized delivery index sequence data. Although multidimensional data contains rich information, it can be complex to analyze and visualize. Dimensionality reduction processing uses certain algorithms to convert multidimensional data into low-dimensional data while preserving as many of the main features of the original data as possible. For example, methods such as principal component analysis (PCA) can be used to extract three main components from the multidimensional normalized delivery index sequence data and convert it into three-dimensional normalized delivery index sequence data.

[0196] In this embodiment, the 3D effect coordinate difference between adjacent time points is calculated based on the 3D normalized delivery index sequence data, and a coordinate difference sequence is generated: the 3D normalized delivery index sequence data is regarded as a sequence of points in 3D space, with each time point corresponding to a 3D coordinate. For two adjacent time points, the difference between their 3D coordinates is calculated, that is, the difference between the coordinate values ​​in the three dimensions is calculated respectively.

[0197] In this embodiment, the first derivative of the coordinate difference sequence is performed to obtain the effect change rate curve: the coordinate difference sequence is treated as a function of time, and its first derivative is calculated. By calculating the derivative for each data point, for example using numerical differentiation methods such as the central difference method, the effect change rate at each time point is obtained. Connecting these change rates forms the effect change rate curve.

[0198] In this embodiment, all effect warning time points within the currently collected period are identified based on the effect change rate curve: the effect warning time point is determined by setting a certain threshold or observing abrupt changes in the curve. For example, if the effect change rate suddenly increases or decreases beyond a certain set threshold (e.g., the change rate exceeds 0.5 or is less than -0.5), it is considered that an abnormal situation may have occurred at that time point, which is an effect warning time point.

[0199] In this embodiment, the differences in the three-dimensional effect coordinates and the differences in the brand identification feature weights of all effect warning time points are mapped to a two-dimensional optimization space to obtain multiple two-dimensional data points. For each effect warning time point, its corresponding three-dimensional effect coordinate difference and brand identification feature weight difference (i.e., the difference in the weights of each feature when identifying the brand before and after the change in the campaign effect) are converted into points in a two-dimensional space through a certain mapping rule. For example, the X-axis represents the weight deviation value, and the Y-axis represents the effect deviation value, resulting in multiple two-dimensional data points.

[0200] In this embodiment, a quadratic polynomial is used to fit all data points in the two-dimensional optimization space to obtain a fitted curve. The cause of fluctuations in the delivery performance is determined based on the second derivative of the fitted curve. Using methods such as least squares, a quadratic polynomial function is found that approximates all data points in the two-dimensional optimization space as closely as possible, and its second derivative is calculated. The sign and magnitude of the second derivative reflect the concavity and curvature of the curve, thus helping to determine the cause of fluctuations in the delivery performance. For example, if the second derivative is greater than 0, the curve is convex downwards, which may indicate that the fluctuation in the delivery performance is due to a gradually increasing factor; if the second derivative is not greater than 0, the curve is convex upwards, which may mean that a gradually weakening factor is affecting the delivery performance.

[0201] In this embodiment, the current multi-dimensional weights are adjusted based on the reasons for fluctuations in campaign performance and the corresponding second derivatives, and a performance summary is generated: Based on the reasons for campaign performance fluctuations analyzed through the second derivative of the fitted curve, and combined with the specific values ​​of the second derivatives, the current multi-dimensional weights used for brand identification are adjusted. For example, if it is found that the fluctuation in campaign performance is caused by an excessively high weight for a certain feature, the weight of that feature is appropriately reduced. After adjusting the multi-dimensional weights, a performance summary is generated. The performance summary will summarize the campaign performance within a preset time period, including changes in various campaign performance indicators, analysis of performance warning time points, reasons for campaign performance fluctuations, and explanations of the multi-dimensional weight adjustments.

[0202] Experimental results show that the present invention achieves a 92% accuracy rate in recognizing commercial videos containing colloquial expressions (such as "pxx" and "Geely car"), which is 53% higher than the existing text keyword matching technology (60%). In multi-brand scenarios, the accuracy rate of core brand recognition reaches 88%, and the average conversion rate of advertising campaigns is increased by 27%.

[0203] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for content extraction and delivery based on multimodal commercial video, characterized in that, include: Call the preset domain-brand mapping library to determine whether the domain information in the mounting link of the commercial video can be directly matched with the brand. If so, output the preliminary brand identifier; otherwise, obtain the metadata of the page pointed to by the mounting link and extract brand-related clues as brand information to be verified. When the domain information in the embedded link of the commercial video cannot be directly matched with the brand, the corresponding content extraction strategy is executed based on the commercial type, and multi-dimensional weights are combined to obtain targeted multimodal features. Based on the initial brand identity or targeted multimodal features, the brand information to be verified, and the brand vector library, the core brand is determined, and the core brand is standardized to obtain a standardized brand identity. Based on standardized brand identity, and combined with the characteristics of the platform to which the commercial video belongs and target user profile data, the system calls upon the advertising strategy library to generate a targeted advertising strategy adapted to the brand type. The commercial video is then pushed to the target user group according to the targeted advertising strategy. Simultaneously, the system monitors advertising performance metrics in real time to obtain monitoring data, adjusts the current multi-dimensional weights based on the monitoring data, and generates a performance summary, including: Multiple campaign performance indicators are collected as monitoring data based on a preset time period, and the sequence records of each campaign performance indicator in the monitoring data are normalized to obtain multidimensional normalized campaign indicator sequence data. Dimensionality reduction processing is performed on the multidimensional normalized delivery indicator sequence data to obtain three-dimensional normalized delivery indicator sequence data; The three-dimensional effect coordinate difference between adjacent time points is calculated based on the three-dimensional normalized delivery index sequence data, and a coordinate difference sequence is generated. The first derivative of the coordinate difference sequence is then calculated to obtain the effect change rate curve. Based on the effect change rate curve, identify all effect warning time points within the current collection period; Map the differences in the three-dimensional effect coordinates of all effect warning time points with the differences in the brand identification feature weights to a two-dimensional optimization space to obtain multiple two-dimensional data points; A quadratic polynomial is used to fit all data points in the two-dimensional optimization space to obtain a fitting curve. The second derivative of the fitting curve is used to determine the reasons for the fluctuation of the campaign effect. Based on the reasons for the fluctuation of the campaign effect and the corresponding second derivative, the current multidimensional weights are adjusted, and a performance report is generated.

2. The method for content extraction and delivery based on multimodal commercial video according to claim 1, characterized in that, The system calls a pre-defined domain-brand mapping library to determine if the domain information in the embedded link of the commercial video can directly match the brand. If so, it outputs a preliminary brand identifier; otherwise, it retrieves the metadata of the page pointed to by the embedded link and extracts brand-related clues as brand information to be verified, including: Obtain the link to the commercial video and extract the domain information from the link; The system calls a pre-defined domain-brand mapping library to determine whether the domain information in the product video's embedding link can directly match the brand. If a direct match is possible, output the initial brand identifier; If a direct match is not possible, obtain the HTML metadata of the linked page through a headless browser, extract brand-related clues from the metadata, and use them as brand information to be verified.

3. The method for content extraction and delivery based on multimodal commercial video according to claim 1, characterized in that, Based on the order type, a corresponding content extraction strategy is executed, and targeted multimodal features are obtained by combining multidimensional weights, including: If the order type is only video order, then adaptive frequency video frame extraction, audio extraction, OCR recognition and removal of colloquial ambiguity are performed to obtain image features, audio features and text features, and combined with multi-dimensional weights to obtain directional multimodal features. If the order type is video order or image / text order, then adaptive frequency video frame extraction, audio extraction, OCR recognition and removal of colloquial ambiguity are performed to obtain image features, audio features and text features. At the same time, image / text titles, body descriptions, cover images, pinned comments and removal of colloquial ambiguity are performed to obtain image / text features and interactive comment features, and then combined with multidimensional weights to obtain directional multimodal features.

4. The method for content extraction and delivery based on multimodal commercial video according to claim 3, characterized in that, The process of performing adaptive frequency video frame extraction to obtain image features includes: Extract industry keywords from the brand information to be verified, and use the industry frame-sampling benchmark frequency table based on the industry keywords to determine the initial frame-sampling frequency. Collect the first n seconds of the video of the commercial order, and calculate the edge pixel ratio of each frame to generate an edge pixel ratio sequence; Calculate the first derivative sequence of the edge pixel proportion sequence, and count the total number of elements in the first derivative sequence of the edge pixel proportion sequence that are greater than the preset first derivative threshold. The image detail features of the commercial video are determined based on the total number of elements in the first derivative sequence of the edge pixel proportion sequence that are greater than the preset first derivative threshold. The adjustment coefficient of the initial frame sampling frequency is determined based on the image detail features, and the initial frame sampling frequency is adjusted based on the adjustment coefficient to obtain the current frame sampling frequency. Based on the current frame extraction frequency, perform video frame extraction on the commercial video to obtain multiple extracted frame images; For each frame-by-frame image, extract the proportion of the logo area, the proportion of the brand-specific color, and the matching degree between the scene and the brand. Based on the proportion of the logo area, the proportion of the brand-specific color, and the matching degree between the scene and the brand for each frame-by-frame image, generate the feature coordinates of each frame-by-frame image. The mean coordinates of the feature coordinates of all extracted frames are used as the image features.

5. The method for content extraction and delivery based on multimodal commercial video according to claim 1, characterized in that, The core brand is determined based on preliminary brand identity or targeted multimodal features, brand information to be verified, and a brand vector library, including: If a preliminary brand identity is obtained, the preliminary brand identity is vectorized and matched with the cosine similarity of each brand standard feature vector in the brand vector library. If the match is successful, the core brand is determined. If directional multimodal features are obtained, the platform's proprietary AI semantic model is invoked. The directional multimodal features and the brand information to be verified are combined to obtain a standardized vector of semantic-level brand information. This vector is then matched with the brand vector library for similarity. If the match is successful, the core brand is determined.

6. The method for content extraction and delivery based on multimodal commercial video according to claim 5, characterized in that, The platform's proprietary AI semantic model is invoked, and standardized vectors of semantic-level brand information are obtained by combining targeted multimodal features and brand information to be verified. These vectors are then matched with a brand vector library for similarity. If a match is successful, the core brand is identified, including: The directional multimodal features are decomposed into three-dimensional coordinates of each feature dimension, and the brand information to be verified in the directional multimodal features is mapped to three-dimensional coordinates to obtain the coordinates of the brand information to be verified. The fusion weight of each feature dimension is calculated based on the reciprocal of the Euclidean distance between the three-dimensional coordinates of each feature dimension and the coordinates of the brand information to be verified. The fused feature coordinates are obtained by weighting and summing the three-dimensional coordinates of each feature dimension based on the fusion weights of each feature dimension. The fused feature coordinates are input into the platform’s proprietary AI semantic model to obtain a standardized vector of semantic-level brand information. Calculate the cosine similarity between the standardized vector of semantic-level brand information and the standard feature vector of each brand in the brand vector library. Then, gather all brands in the brand vector library whose cosine similarity with the standardized vector of semantic-level brand information is not less than a preset similarity threshold to obtain a candidate brand set. At the same time, determine that the match is successful. The evaluation coordinates for each candidate brand are constructed based on its historical identification accuracy, campaign conversion rate, and industry matching degree. From the candidate brand set, the candidate brand with the smallest Manhattan distance between the corresponding evaluation coordinates and the fusion feature coordinates is selected as the core brand.

7. The method for content extraction and delivery based on multimodal commercial video according to claim 5, characterized in that, Also includes: If the current commercial video is set in a multi-brand scenario, then a three-level conflict resolution strategy will be initiated to identify the core brand.

8. The method for content extraction and delivery based on multimodal commercial video according to claim 5, characterized in that, Also includes: If the match fails, the corresponding preliminary brand identifier or semantic brand information will be stored in the new brand review pool until manual review is received. Then, the corresponding preliminary brand identifier or semantic brand information will be updated in the brand vector library and domain-brand mapping library.

9. The method for content extraction and delivery based on multimodal commercial video according to claim 1, characterized in that, Based on standardized brand identity, and combined with the characteristics of the platform to which the commercial video belongs and target user profile data, the system calls upon the advertising strategy library to generate a targeted advertising strategy adapted to the brand type, and pushes the commercial video to the target user group according to the targeted advertising strategy, including: The standardized brand logo is assigned a brand type value according to the brand type assignment rules, the platform traffic conversion capability is assigned a value based on the characteristics of the platform to which the commercial video belongs, and the matching degree between the user and the core brand is assigned a value based on the target user profile data. Based on brand type assignment, platform traffic conversion capability assignment, and user-core brand matching assignment, a coordinate system for the ad placement strategy is constructed. The benchmark strategy with the smallest Euclidean distance between the corresponding benchmark strategy coordinate and the corresponding ad placement strategy coordinate is selected from the ad placement strategy library and used as the target strategy. The target strategy is then fine-tuned based on the Euclidean distance to obtain a targeted ad placement strategy that is suitable for the brand type.

Images