A multi-modal marketing material copyright verification method

Abstract

The application relates to the technical field of digital content processing, and discloses a multi-modal marketing material copyright verification method; the method first receives to-be-verified marketing materials and corresponding metadata, verifies material file formats, delivery information and authorization verification fields, and establishes a verification task record; images, texts, fonts, formats, videos, subtitles and audios in the materials are subjected to hierarchical analysis, a material correlation graph is constructed, and a combination feature unit is extracted; candidate copyright assets are searched according to the combination feature unit, overall features of the materials and historical review records; the contents, structures, styles and audio-visual relationships of the candidate copyright assets are reviewed; the authorization state records of the candidate copyright assets are called, the review contents are matched with delivery time, delivery area, delivery platform and use mode, and a risk verification record is formed; the method can improve the accuracy and traceability of multi-modal marketing material copyright verification.

Description

Technical Field

[0001] This invention relates to the field of digital content processing technology, specifically to a method for verifying the copyright of multimodal marketing materials. Background Technology

[0002] With the development of digital marketing, advertising materials are no longer just single images or text, but rather a combination of images, copy, fonts, layouts, video clips, subtitles, and background music. Before being published, distributed, or reused, materials typically need to be verified to ensure they do not resemble existing copyrighted materials and to determine if the current usage falls within the scope of authorization. For example, authorized Chinese invention patent CN111182364B discloses a short video copyright detection method and system, which mainly uses image frame vector indexing and the number of consecutive similar image frames to determine whether a short video has copyright risks. Another example is authorized Chinese invention patent CN111625779B, which discloses a blockchain-based method for protecting the copyright of digital content on social networks. This method mainly uses blockchain to record the uploading, publishing, forwarding, and querying processes of digital works to achieve copyright protection and traceability of the dissemination path of digital content. While the aforementioned existing technologies can play a role in short video similarity detection or digital content flow tracing, they still have shortcomings when marketing materials consist of images, text, audio, fonts, and layout. Specifically, existing verification methods often focus on feature comparison of single content types, easily severing the connections between text and visual meaning, font style and color matching, and video rhythm and background music tempo. This makes it difficult to promptly identify materials with low similarity in individual content elements but similar overall creative expression. Furthermore, common modifications such as image cropping, color correction, text rewriting, replacement of similar fonts, and changes in background music tempo weaken the stability of single feature comparisons, leading to… The recall of similar candidate materials is insufficient. Furthermore, existing methods often struggle to accurately correlate similarity detection results with information such as authorization period, authorized region, distribution platform, usage method, and secondary creation permission. Consequently, authorized materials are easily misjudged as risky materials, or expired or out-of-scope materials are not promptly identified. Therefore, there is a need to provide a copyright verification method applicable to multimodal marketing materials that comprehensively verifies different types of material content and their interrelationships, and assesses actual risks by combining the scope of authorization and usage scenarios. This would improve the accuracy of similar material recall, the accuracy of out-of-scope authorization identification, and the efficiency of review and processing. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a method for verifying the copyright of multimodal marketing materials, which solves the problems mentioned in the background section.

[0004] To achieve the above objectives, the present invention provides the following technical solution: A method for verifying the copyright of multimodal marketing materials includes: S1: Receive marketing materials to be verified and corresponding metadata, verify the material file format, placement information and authorization verification fields, and establish a verification task record; S2: Perform layered analysis on the marketing materials to be verified, and identify image areas, text areas, font samples, layout structure, video clips, subtitle content and appearance time, and audio clips; S3: Based on the hierarchical analysis results, establish a material relationship diagram and extract combined feature units between images, text, fonts, layouts, videos, and audios; S4: Based on the combined feature units and the overall features of the material, retrieve candidate copyright assets in the copyright asset library, and review the content, structure, style and audiovisual relationship of the candidate copyright assets; S5: Call the authorization status record of the candidate copyright assets, match the review content with the release time, release area, release platform and usage method, and generate risk verification record.

[0005] Preferably, S1 includes: Material files and metadata can be obtained through the material submission interface, batch import interface, pending review queue, or creator-side channel; Verify the file header, encoding format, resolution, frame rate, sampling rate, subtitle duration, compressed file directory, delivery field, and authorization field; Establish a task index based on the receiving time, uploader identifier, and file content summary value; Set flags for missing fields, unparseable files, inconsistent frame rates, audio unparsed, or abnormal credentials.

[0006] Preferably, a task index is established based on the receiving time, the uploader identifier, and the file content digest value, including: Standardize the receiving time into a unified timestamp; Associate the uploader identifier, file content summary value, and verification task number; Generate sub-file numbers for image files, video files, audio files, subtitle files, and description files within the same resource package; Write the sub-file number, material type, and source file information into the index record.

[0007] Preferably, S2 includes: Select the full, lightweight, or exception rollback parsing method based on the material type and the list of parsable files; Perform region segmentation, confidence level marking, and layout positioning on images or posters; Extract the glyphs, size, and color of the text and create font samples; Divide the video into segments based on camera transitions, subtitle segments, or time windows; Write the object type, location coordinates, time interval, source file, and confidence level of each parsed object into the hierarchical parsing record.

[0008] Preferably, based on the material type and the list of parseable files, a full, lightweight, or anomaly rollback parsing method is selected, including: Read the material type, file format verification results, and list of parsable files from the verification task record; Full analysis of pre-release review task execution; Lightweight analysis of early warning tasks executed on the creation side; When a video has no audio, an image has no text, or a subtitle file is corrupted, an abnormal rollback parsing is performed, and the video is processed according to the parsable image, text, subtitle, or audio modality.

[0009] Preferably, S3 includes: Set the parsed object in the hierarchical parsing record as an object node; Define semantic correspondence, location correspondence, time synchronization, style matching, and source correspondence as relation edges; Establish relationships between object nodes based on spatial proximity, thematic keywords, font size hierarchy, color difference values, and time overlap; Set an anomaly flag for low-confidence objects, misaligned subtitle timelines, or missing audio. Combined feature units are extracted from the correspondence between text and images, the correspondence between font shapes, the correspondence between layouts, the correspondence between subtitles, and the correspondence between audiovisual elements.

[0010] Preferably, S4 includes: The asset pool to be searched is determined based on material type, platform, brand identity, source of material, and asset tags; A candidate queue is generated based on the combined feature units, the overall features of the material, and historical review records; The text, images, subtitles, audio, layout, fonts, color relationships, and audiovisual timing relationships of the candidate copyright assets were reviewed item by item. Set a review flag when the asset database is inaccessible, the sub-database is missing, the number of candidates exceeds the upper limit, or the matching level is lower than the preset level.

[0011] Preferably, a candidate queue is generated based on combined feature units, overall material features, and historical review records, including: Candidate assets are obtained according to asset types: images, fonts, music, templates, and historical materials. Set the assets whose source descriptions correspond to their asset tags as high-priority candidate assets; Supplement candidate assets based on the characteristics of text and image combination, layout combination, audiovisual combination, and overall material characteristics; Candidate assets are sorted according to historical hit frequency, historical false alarm records, and asset type priority.

[0012] Preferably, S5 includes: Retrieve authorization status records based on candidate asset number; When no direct corresponding record is obtained, the authorization record shall be associated with the asset source platform, asset name, authorization certificate number or historical verification record; Match the authorized time, authorized region, authorized platform, and permitted usage method with the corresponding delivery fields one by one; Set a review flag when the platform name is inconsistent, there are multiple authorization records, authorization status records are missing, or the delivery field is missing; Write the risk verification record according to the review level and matching status.

[0013] Preferably, the authorization time, authorization region, authorization platform, and permitted usage method are matched item by item with the corresponding delivery fields, including: Compare the launch start time and launch end time with the authorized start time and authorized end time respectively within the time interval; The deployment areas and authorized regions are compared and classified. The target delivery platforms are standardized according to the platform name mapping table and then compared with the authorized platforms; Compare the usage of the material with the permitted usage in the authorization status record.

[0014] Compared with existing technologies, this invention provides a method for verifying the copyright of multimodal marketing materials, which has the following beneficial effects: 1. This invention, through document verification, layered analysis, construction of material relationship diagrams, and extraction of combined feature units for marketing materials to be verified, integrates images, text, fonts, layouts, videos, subtitles, and audio, along with their spatial, temporal, and stylistic relationships, into the verification process. By combining the overall characteristics of the materials, combined feature units, and historical review records to retrieve candidate copyright assets, and by conducting item-by-item reviews of the content, structure, style, and audiovisual relationships of the candidate copyright assets, it can comprehensively verify materials that have undergone common modifications such as image cropping, color correction, copywriting rewriting, font replacement, and music tempo changes. By matching the authorization status of candidate copyright assets with the placement time, placement area, placement platform, and usage method item by item, it can distinguish between authorization coverage and authorization overreach situations of similar materials, thereby improving the accuracy of similar material recall, the accuracy of authorization overreach identification, and the efficiency of review and processing.

[0015] 2. This invention establishes verification task records and task indexes during the material receiving stage, and records object type, source file, time interval, confidence level, verification level, matching status, and abnormal fields during hierarchical parsing, relationship building, candidate retrieval, itemized review, and authorization matching processes. This ensures that images, videos, subtitles, audio, and authorization information within the same marketing material package maintain consistent data association. By setting review flags for situations such as missing fields, unparseable files, low-confidence objects, misaligned timelines, missing asset databases, and missing authorization status records, it facilitates abnormal materials entering rollback or manual review processes, thereby improving data continuity, accuracy of abnormal location, and traceability of review records in the verification process. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the process for a multimodal marketing material copyright verification method according to the present invention; Figure 2 This is a schematic diagram illustrating the verification task reception and task index establishment of the present invention; Figure 3 This is a schematic diagram illustrating the layered analysis of multimodal materials in this invention; Figure 4 This is a diagram illustrating the material relationship and extracting combined feature units for this invention. Figure 5 This is a schematic diagram illustrating the retrieval and itemized review of candidate copyright assets for this invention; Figure 6 This is a schematic diagram illustrating the generation of authorization status matching and risk verification records for this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1: Figures 1-6 A method for verifying the copyright of multimodal marketing materials is presented, including: S1: Receive marketing materials to be verified and corresponding metadata, verify the material file format, placement information and authorization verification fields, and establish a verification task record; S2: Perform layered analysis on the marketing materials to be verified, and identify image areas, text areas, font samples, layout structure, video clips, subtitle content and appearance time, and audio clips; S3: Based on the hierarchical analysis results, establish a material relationship diagram and extract combined feature units between images, text, fonts, layouts, videos, and audios; S4: Based on the combined feature units and the overall features of the material, retrieve candidate copyright assets in the copyright asset library, and review the content, structure, style and audiovisual relationship of the candidate copyright assets; S5: Call the authorization status record of the candidate copyright assets, match the review content with the release time, release area, release platform and usage method, and generate risk verification record.

[0019] This method can be applied to pre-release review platforms for advertising materials, brand material management platforms, e-commerce platform marketing material review systems, short video platform content review systems, commercial material authorization platforms, or internal enterprise compliance review platforms. The executing entity can be a server, cloud computing device, edge computing device, content review terminal, or enterprise data processing device with data processing and information interaction capabilities. The marketing materials to be verified can be static posters, combined graphic and text advertisements, short video advertisements, multimedia materials with subtitles and background music, dynamic banner materials, or promotional materials containing AI-generated content. The verification data received by the electronic device includes at least the marketing materials to be verified, material metadata, authorization certificate information, and business scenario information. The device receives, parses, associates, retrieves, reviews, and performs authorization matching processing on the aforementioned verification data, thereby forming verification task records, hierarchical analysis records, material relationship diagrams, combined feature units, candidate copyright asset review records, authorization matching records, and risk verification records. The confidence level, time window, number of candidates, matching level, and review score involved in this method can all be used as initial thresholds. The aforementioned initial thresholds can be determined based on the specifications of the material files, common platform encoding formats, historical manual review samples, task processing time limits, and the size of the copyright asset library. They can also be calibrated based on the manual review records of the most recent 30, 90, or 180 days. Among them, the 30-day sample is suitable for scenarios with rapid updates, such as short videos and trending marketing materials; the 90-day sample is suitable for regular graphic and text advertising materials; and the 180-day sample is suitable for copyright assets with longer update cycles, such as fonts, templates, and background images. When electronic devices adjust the thresholds, the values ​​before and after the adjustment, the sample time range, the reason for the adjustment, and the effective time are recorded so that the thresholds can correspond to the material specifications, asset library size, and historical review results, avoiding the use of fixed parameters under different material types and different asset library sizes, which would affect the stability of the verification.

[0020] Specifically, such as Figure 2As shown: When receiving and verifying marketing materials to be verified, the electronic device receives the marketing materials to be verified and their corresponding metadata through the verification task entry. The verification task entry can be a material submission interface, a batch import interface, a platform pending review queue, or a creator's submission channel. The marketing materials to be verified can be image files, video files, audio files, image and text combination files, or compressed files. The file format can be JPG, PNG, WEBP, MP4, MOV, MP3, WAV, PDF, PSD exported files, or platform internal material package formats. The corresponding metadata includes at least the following fields: upload subject identifier, material name, material type, target placement platform, placement area, placement start time, placement end time, usage method, material source description, authorization certificate number, and business scenario identifier. After receiving the above data, the electronic device generates a verification task number for each marketing material to be verified and establishes a task index based on the reception time, upload subject identifier, and file content summary value or content summary value, so that hierarchical parsing, candidate retrieval, review processing, and authorization matching under the same task can refer to a unified task record. When electronic devices perform format verification on media files, they check the consistency of file extensions, header information, encoding format, file size, resolution, video frame rate, audio sampling rate, subtitle timing information, and compressed package directory structure. For static images, the minimum resolution is preferably no less than 300 pixels by 300 pixels. This value is based on the need to retain basic stroke information for text region recognition and font outline extraction. When the image size is smaller than this range, common Chinese character strokes, logo edges, and small-sized explanatory text are prone to breakage, adhesion, or overlap, which will reduce the stability of subsequent text recognition and font sample extraction. For mobile vertical posters or short video covers, the resolution verification reference value can be selected as 720 pixels by 1280 pixels or 1080 pixels by 1920 pixels. This range corresponds to the vertical screen specifications of common mobile marketing materials and can meet the basic clarity requirements for image area, text area, and layout structure analysis. For horizontal materials, square image materials, or platform-customized size materials, they can be adjusted proportionally according to the short side pixels and aspect ratio. For video materials, the default verification duration can be selected from 1 second to 300 seconds. Videos shorter than 1 second are usually difficult to form a stable correspondence between the image, subtitles, and audio. Videos longer than 300 seconds are likely to contain multiple creative scenes or multiple advertising segments. They can be segmented for verification according to shot transitions, subtitle segments, or fixed time windows to reduce the amount of data per task and reduce the occurrence of different scenes being mixed into the same verification segment. For audio data, the sampling rate verification range can be selected from 16kHz to 48kHz. 16kHz can cover the basic recognition requirements of speech audio, while 44.1kHz and 48kHz are common sampling rate ranges in music materials and audio encoding on video platforms. Therefore, this range can adapt to the speech, background music, and short video audio in marketing materials. If the marketing material to be verified is a silent video, the electronic device will record no audio identifier and will skip audio segment processing in subsequent layered analysis and audiovisual relationship verification. When verifying the delivery information, the electronic device reads fields such as the target delivery platform, delivery region, delivery time, and usage method, and checks whether the fields are missing, whether the field content matches the platform's preset field table, and whether the time format meets the unified time rules. The delivery time can be converted into a standard timestamp. For cross-regional delivery materials, the local time of the submitting end and the unified time of the platform can be retained in the task record. The delivery region can be represented by country, province / city, platform region code, or business-defined region code, preferably using a hierarchical region field to facilitate subsequent matching with the authorized geographical scope. The usage method should at least include advertising delivery, product detail page display, short video release, live room display, offline material printing, secondary adaptation, or cross-border delivery. If the usage method field is missing, the electronic device does not directly terminate the verification, but marks the field as pending supplementation and generates a pending review record using conservative matching rules during subsequent authorized matching. When the electronic device verifies the authorization verification fields, it checks fields such as the source description of the material, the authorization certificate number, the commercial material platform asset number, the font authorization number, the music authorization number, the template authorization number, and the internal proprietary material number. If the authorization certificate number exists but its format does not conform to the platform's authorization certificate rules, the electronic device records an abnormal certificate format flag and writes the flag into the verification task record. If the authorization field is empty but the source of the material is marked as proprietary material, the electronic device further reads the internal material number or the historical release record number. If the corresponding number is still not read, it records an insufficient source evidence flag. For compressed package materials, the electronic device reads the package's list file and creates sub-file records for images, videos, audio, font files, and description files, so that subsequent layered parsing can locate the specific file and its source. Depending on the application scenario, the receipt and verification of marketing materials to be verified can adopt synchronous pre-review, asynchronous batch processing, or real-time submission from the creator. Synchronous pre-review is suitable for single-material review scenarios before release. After receiving the material, the electronic device completes format verification, field verification, and task record creation. If the verification fails, it returns a prompt to supplement fields or re-upload. Asynchronous batch processing is suitable for scenarios where brands or advertising agencies submit materials in batches. The electronic device first creates a batch task record, and then verifies each material in the queue. The failure of a single material verification does not affect the continued processing of other materials in the same batch. Real-time submission from the creator is suitable for design tools or creator platforms. When the user saves or exports the material, the electronic device receives the current material snapshot and temporary metadata, and enters the subsequent parsing in the form of a draft task. The placement information and authorization fields are then supplemented before release. Regarding anomaly handling, if the source file is corrupted, the file header is unrecognizable, or the file size is 0, the electronic device will mark the task status as unparseable and stop subsequent parsing processing. If only some fields are missing, the electronic device will retain the parsable content and set a supplementary flag for the missing fields. If the video frame rate is inconsistent with the metadata record, the electronic device will use the actual parsed frame rate of the file as the standard, while retaining the metadata anomaly flag. If the audio sampling rate cannot be read but the video track can be parsed, the electronic device will set the audio segment to a pending parsing status. After the above processing, the verification task record will include at least the task number, file verification result, field integrity flag, standardized delivery information result, and authorized field verification flag. The above records serve as the data basis for subsequent hierarchical parsing path selection and anomaly rollback processing.

[0021] Specifically, such as Figure 3 As shown: During the layered parsing process, the electronic device performs layered processing on the marketing materials to be verified based on the material type, file format verification results, and list of parsable files in the verification task record. Layered parsing does not perform single content recognition on the materials, but extracts data such as images, text, fonts, layout, videos, subtitles, and audio separately, and writes all types of data into a unified layered parsing record, so that the subsequent material relationship diagram can directly call the same data objects. For static images or poster-type materials, the electronic device reads the image resolution, color space, and pixel content, and identifies the background area, main image area, brand logo area, decorative element area, and text area. For combined graphic materials, the electronic device reads information such as group boundaries, page order, component position, and component size. For video materials, the electronic device reads basic video information, key shots, screen size, frame rate, shot transition positions, subtitle content, and audio data in chronological order. For materials containing independent subtitle files, the electronic device reads the text content, start time, and end time of the subtitle files. For materials where subtitles are directly superimposed on the video screen, the electronic device extracts the subtitle content through on-screen text recognition and determines the subtitle appearance time by combining the duration of text appearance in adjacent frames. During image region recognition, electronic devices can employ target region detection, salient region extraction, or region segmentation based on layout rules. For regular-format posters, image regions can be divided according to edges, white space, color blocks, and text arrangement. For complex composite posters, the main object, background, logo, and decorative regions can be identified first, and then the coordinates of the top left corner, width, height, center point, level, and confidence level of each region can be recorded. The effective confidence level of the object region can be selected to be above 0.75. This value is based on the trade-off between the false positive rate and false negative rate of objects in historical manually reviewed samples. For materials with high clarity and regular layout, the confidence threshold can be increased to 0.80. For low-resolution materials or materials with significant occlusion, the confidence threshold can be reduced to 0.70, and the corresponding objects are marked as low-confidence objects, reducing their priority in subsequent relationship establishment and review. Regions below the threshold are not directly deleted but are added to the low-confidence region list to avoid the direct discarding of semi-transparent decorations, gradient backgrounds, or occluding elements, which would lead to incomplete subsequent association relationships. During the text region and font sample recognition process, the electronic device extracts text regions from images or videos and identifies information such as text content, font size range, character outline, thickness, slant, character spacing, line spacing, and text color. When the text recognition confidence score is below 0.80, the electronic device enters character sampling backtracking processing. This value is higher than the effective confidence score of the object region because the text recognition result will simultaneously affect text review, subtitle content correspondence, and font sample grouping. When the text recognition confidence score is insufficient, retaining the character outline, stroke width, character height, character boundary, and text color is more beneficial for subsequent review than directly using low-confidence text content. If there are multiple fonts in the material, the electronic device groups the font samples according to the text region position, font appearance, and font size range, and records the text region and occurrence position corresponding to each group of font samples. During the layout structure recognition process, the electronic device reads the relative positional relationships between image areas, text areas, brand logo areas, and decorative element areas to form a layout structure record. The layout structure can be represented using a fixed grid, an adaptive grid, or object bounding box relationships. The fixed grid side length can be selected from 32 pixels to 64 pixels, which is suitable for common mobile posters or video cover materials with a short side of approximately 720 pixels to 1080 pixels, balancing the accuracy of text block positioning and the integrity of the layout structure. If the material resolution is higher or lower than this range, the grid side length can be adjusted proportionally to the short side length of the material to keep the number of units after grid division within a range that can express the main text blocks, image blocks, and logo areas. The adaptive grid is suitable for posters with a large main visual area and fewer text layers. The electronic device determines the layout partitions based on the boundaries of significant areas and blank intervals. Object bounding box relationships are suitable for key video shots and complex graphic templates. The electronic device records the top-bottom, left-right, overlapping, adjacent, centered, and aligned relationships between objects. During the identification of video clips, subtitle content, appearance time, and audio segments, electronic devices divide the video into multiple segments based on the video frame rate and shot transition positions. For short videos with a total duration of no more than 30 seconds, the video segment unit can be selected from 3 to 5 seconds, which can usually cover an advertising shot or a subtitle expression segment. For videos with a total duration of more than 30 seconds but no more than 300 seconds, the video segment unit can be selected from 5 to 8 seconds to reduce the number of segments and retain the integrity of the shot segments. The above time range is derived from the shot transition rhythm and subtitle presentation duration of common marketing short videos, which can avoid semantic incompleteness due to excessively short segments and also avoid multiple creative scenes being mixed due to excessively long segments. If the actual shot transition point or subtitle segment boundary is inconsistent with the fixed time window, the segmentation can be prioritized based on the shot transition point and subtitle segment boundary. The content and appearance time of subtitles can be determined by the subtitle file or the duration of continuous text appearance on screen; audio segments can be divided based on silence boundaries, stable beat intervals, or video segment boundaries; if there is a time axis offset between audio and video, the electronic device uses the video start time as a unified reference to convert the start and end times of audio segments to the same time axis; the audio sampling rate can be selected from 16kHz, 24kHz, 44.1kHz, or 48kHz for subsequent rhythm auditory verification; among them, 16kHz is suitable for speech audio, 44.1kHz and 48kHz are common sampling rates in music materials and video platform audio encoding, and 24kHz can be used for intermediate processing scenarios that balance speech recognition and computational resource consumption; if the original sampling rates are different, they can be uniformly processed, and the sampling rates before and after conversion can be recorded; Depending on the material review scenario and device processing resources, layered parsing can adopt full parsing, lightweight parsing, or anomaly rollback parsing. Full parsing is suitable for strict pre-release review, where electronic devices fully recognize images, text, fonts, layouts, videos, subtitles, and audio. Lightweight parsing is suitable for real-time alerts on the creation end, where electronic devices prioritize recognizing image areas, text areas, layout structures, and audio segments, while temporarily suspending time-consuming frame-by-frame video processing. Anomaly rollback parsing is suitable for scenarios where some modalities are missing or parsing fails, such as skipping audio segment recognition when there is no audio or video, skipping text area and font sample recognition when there is no text in the image, and using on-screen text recognition to extract subtitle content and appearance time when the subtitle file is corrupted. After hierarchical parsing, the electronic device generates a hierarchical parsing record. The hierarchical parsing record includes at least the following fields: object type, object number, location coordinates, time interval, source file, confidence level, and associated task number. For static materials, the time interval can be empty or uniformly marked as static. For video materials, the time interval corresponds to the video clip, subtitle appearance time, and audio clip. This hierarchical parsing record serves as the basic data for subsequently establishing a material relationship graph, enabling image regions, text regions, font samples, layout structures, video clips, subtitle content and appearance time, and audio clips to be directly referenced in subsequent processing without re-parsing the original file.

[0022] Specifically, such as Figure 4 As shown: When establishing the material relationship diagram, the electronic device reads the spatial, temporal, and content relationships between image regions, text regions, font samples, layout structures, video clips, subtitle content and appearance time, and audio clips based on the hierarchical parsing records. It then establishes verifiable data relationships between different objects in the same marketing material. The material relationship diagram is used to represent the connections between different material objects, so that images, text, fonts, layouts, videos, and audios are no longer used as independent single features in subsequent verification, but rather as material objects with interrelationships in candidate retrieval and verification. The material relationship diagram can be composed of object nodes and relationship edges. Object nodes correspond to specific parsed objects in the hierarchical parsing record, including at least image object nodes, text object nodes, font sample object nodes, layout structure object nodes, video clip object nodes, subtitle object nodes, and audio object nodes. Relationship edges are used to record semantic correspondence, positional correspondence, time synchronization, style matching, or source association between objects, including at least image-text correspondence edges, text-font correspondence edges, text-layout correspondence edges, video frame-subtitle time relationship edges, video frame-audio time relationship edges, image region-layout structure relationship edges, and audio clip-subtitle content relationship edges. When establishing a correspondence between text and images, the electronic device reads the text content of the text area and the image objects of the image area, and establishes candidate correspondences based on the thematic consistency between the text content and the image objects. For example, when a product name, function description, promotional copy, or brand phrase appears in the text area, the electronic device establishes a candidate correspondence between the text area and the nearby main image area, brand logo area, or decorative element area. Candidate correspondences can be determined jointly by location proximity rules, content keyword rules, and area hierarchy rules. The proximity rule compares the distance from the center point of the text area to the center point of the image area with the length of the material's diagonal. When this distance accounts for no more than 0.35 of the material's diagonal length, electronic devices prioritize establishing candidate correspondences. This ratio can be adjusted between 0.25 and 0.45, where 0.25 is suitable for posters with the main subject centered and text arranged around the main image, and 0.45 is suitable for horizontal or vertical layouts. This range is derived from the characteristic that the main title, main visual, and brand logo are usually located in the main expression area in advertising posters. If the ratio is too large, it is easy to mistakenly associate the auxiliary text on the edge with the main visual area; if the ratio is too small, it is easy to miss the text-image correspondence in the column layout. The content keyword rule can match product names, brand words, scene words in the copy with image recognition tags. The area hierarchy rule can determine the primary and secondary relationships based on font size, text area area, and visual prominence. Text areas with larger font sizes, centered, or close to the main object are prioritized to establish a relationship with the main image area. When establishing relationships between fonts, layouts, and styles, electronic devices combine font samples, text colors, font size ranges, row and column arrangements, and the layout structure corresponding to the same text area. If multiple text areas use the same or similar fonts and are arranged along the same layout axis, the electronic device associates these text areas with the same layout structure object. For multi-font posters, electronic devices can establish hierarchical relationships according to title fonts, body text fonts, emphasis fonts, and supplementary explanatory fonts. The relationship between font samples and colors can be determined based on the color difference value between the text color and the background main color, decorative color, or brand color. The color difference value can be obtained under a unified color space. Based on the magnitude of the color difference value, the relationship between font samples and colors can be recorded as strong association, weak association, or no association to avoid judging the coordination relationship between fonts and colors based solely on a single color value. When establishing the relationship between video, subtitles, and audio, the electronic device reads the start and end times of the video segment, the appearance time of the subtitle content, and the start and end times of the audio segment according to a unified timeline. If the overlap ratio between the appearance time of the subtitle and the time of the video segment reaches 0.60 or higher, the electronic device establishes a time correspondence between the subtitle content and the corresponding video segment. The basis for setting this ratio is that short video subtitles usually appear earlier or disappear later than the on-screen action, and an overlap ratio of 0.60 or higher can accommodate the common situation where subtitles cross the shot boundary. For materials with high requirements for synchronization between subtitles and the screen, this ratio can be increased to 0.70 or higher. If the stable tempo of an audio clip is adjacent to the shot transition time, and the time difference between the two is no more than 0.5 seconds, the electronic device will establish an audiovisual correspondence between the audio clip and the video clip. 0.5 seconds corresponds to a time range of approximately 15 frames per second for a 30-frame-per-second video and approximately 30 frames per second for a 60-frame-per-second video. This range can cover the normal editing errors that exist between the shot transition and the beat point in short video editing. If the frame rate of the material is low or the editing rhythm is slow, this time difference can be appropriately widened. If the material is used for fast-paced music advertisements or beat-matching video review, this time difference can be appropriately narrowed. After the material relationship diagram is established, the electronic device extracts combined feature units from the material relationship diagram. A combined feature unit consists of at least two object nodes and one relationship edge, used to record the relationship information that a single material object cannot express on its own. Combined feature units include at least image-text combined feature units formed by image regions and text regions, glyph combined feature units formed by text regions and font samples, layout combined feature units formed by text regions and layout structures, subtitle combined feature units formed by video clips, subtitle content and appearance time, and audiovisual combined feature units formed by video clips and audio clips. For complex materials, combined feature units can consist of multiple object nodes and multiple relationship edges. For example, the main image region, title text region, font sample and layout structure can jointly form the main visual combined feature unit. The material relationship diagram can be formed through rule-based relationship establishment, recognition result-assisted relationship establishment, or manual annotation and write-back. Rule-based relationship establishment is suitable for templated advertisements or materials with stable field structures. Electronic devices establish object relationships based on spatial distance, temporal overlap, font size hierarchy, color relationships, and object type tables. Recognition result-assisted relationship establishment is suitable for complex composite materials. Electronic devices read semantic recognition results and visual recognition results based on rule judgments to supplement the correspondence between image and text themes. Manual annotation and write-back is suitable for controversial or high-value materials. Human reviewers can confirm the relationships between image areas, text areas, font samples, layout structures, video clips, subtitle content, appearance time, and audio clips in the review interface. The electronic device writes the confirmed relationships back to the material relationship diagram record for subsequent analysis of similar materials. Regarding anomaly handling, if the confidence level of a certain object node is lower than the effective threshold given in the hierarchical parsing stage, the electronic device can participate in relationship establishment with low weight, but will not use it as the basis for triggering high-risk review alone. If the subtitle time is significantly misaligned with the video timeline, the electronic device will first correct it based on the continuous appearance time of the subtitle in the picture. If the correction fails, the subtitle relationship will be marked as a time anomaly. If an audio segment is missing, the electronic device will establish a material relationship graph without audio object nodes and mark the audiovisual relationship review as unavailable. If there are too many object relationships, resulting in a high complexity of the material relationship graph, the electronic device will prioritize retaining relationships related to the main image, core text, brand logo, main font, and main audio segment, and will treat auxiliary decorative relationships as low-priority relationships. The material relationship graph and combined feature units formed by the above processing can serve as the data basis for subsequent candidate copyright asset retrieval and comparison review, so that candidate retrieval no longer relies solely on the similarity comparison of a single image, a single sentence of text, or a single audio segment.

[0023] Specifically, such as Figure 5 As shown: When performing candidate copyright asset retrieval and verification, the electronic device reads the previously formed combined feature units and combines them with image regions, text regions, font samples, layout structure, video clips, subtitle content and appearance time, and audio clips to form the overall feature of the material. The overall feature of the material is used to represent the comprehensive performance of the marketing material to be verified in terms of content, structure, style, and temporal relationship. For static posters, the overall feature of the material is primarily formed by image regions, text regions, font samples, layout structure, and image-text combination feature units. For short video materials, the overall feature of the material is primarily formed by video clips, subtitle content and appearance time, audio clips, and audiovisual combination feature units. For materials without audio, the electronic device does not use audio clips and audiovisual combination feature units to avoid the missing modality interfering with the candidate retrieval. Thus, the overall feature of the material and the combined feature units together serve as the basis for candidate retrieval, enabling the candidate retrieval to cover not only the similarity relationship of individual material objects, but also the correlation relationship such as image-text coordination, layout combination, subtitle synchronization, and audiovisual rhythm. The copyright asset library can include the brand's own material library, a historically published material library, a commercial image licensing library, a commercial font licensing library, a commercial music licensing library, a video template library, a design template library, and a third-party commercial material licensing library. When copyright assets are added to the library, asset feature records can be generated according to the same or corresponding hierarchical parsing rules as the marketing materials to be verified. The asset feature records include at least the following fields: image area, text area, font sample, layout structure, video clip, subtitle content and appearance time, audio clip, material relationship diagram, and combined feature unit. Each copyright asset can also be associated with fields such as asset number, material type, content summary, licensing status record index, and historical review record index. Through the above-mentioned library entry process, the marketing materials to be verified and the assets in the copyright asset library can be compared based on the same type of data object, avoiding inconsistencies in the retrieval basis caused by one side only storing the original file and the other side storing the parsing result. During the candidate retrieval process, the electronic device can first filter the copyright asset library based on the material type and metadata. For example, when the marketing material to be verified is a static poster, the image library, template library, font library, and historical poster library are searched first. When the marketing material to be verified is a short video, the video template library, music library, subtitle template library, historical short video library, and image material library are searched first. When the marketing material to be verified is a combined graphic and text advertisement, the graphic and text template library, commercial image library, and font authorization list are searched first. The electronic device can also narrow the search scope based on the placement platform, industry category, brand logo, historical activity logo, and material source description. If there is a direct correspondence between the material source description and the asset tag in the copyright asset library, the corresponding asset enters the high-priority candidate queue. If there is no direct correspondence, the electronic device searches based on the combined feature units and the overall features of the material to reduce missed recalls caused by relying solely on a single image, a single sentence of text, or a single audio segment. The number of candidates can be determined based on the size of the copyright asset database, the computational resources for review, and the task processing time. For copyright asset databases with fewer than 100,000 entries, the number of candidates can be selected from 20 to 80. This range can cover the main similar candidates in a small-scale asset database while avoiding too many invalid candidates. For copyright asset databases with 100,000 to 10 million entries, the number of candidates can be selected from 50 to 200. This range is used to balance candidate coverage and review load. For copyright asset databases with more than 10 million entries, the number of candidates can be selected from 200 to 500, and a batch review approach can be adopted to reduce the pressure of a single review. The above numbers can be adjusted based on historical false negative rates, false positive rates, and single task processing time. When the false negative rate increases, the number of candidates can be increased appropriately. When the false positive rate or review time increases, the number of low-priority candidates can be decreased appropriately. During the content review process, electronic devices compare the text content, image area content, subtitle content, and audio clip content in the marketing materials to be verified with the candidate copyright assets. The text content review is used to identify whether the copy has been paraphrased, partially deleted, sentence structure adjusted, or keyword replaced. The image content review is used to identify whether the main image, background image, brand logo, and decorative elements have been cropped, scaled, rotated, or color-corrected. The subtitle content review is used to identify whether the subtitle text and its appearance time are similar to the candidate video assets. The audio clip review is used to identify whether the background music clips, beat intervals, or melody clips are similar to the candidate music assets. The above reviews do not directly form a legal judgment, but generate technical review results for subsequent authorization matching and risk recording. Content review, structure review, style review, and audiovisual relationship review can each output a normalized review score, with a score range of 0 to 1. Review scores can be determined based on the degree of content consistency, positional consistency, temporal overlap, similarity of font samples, similarity of layout structure, and stability of audiovisual correspondence, with different priorities set according to the material type. A strong match score can be selected above 0.80, a medium match score between 0.50 and 0.80, and a weak match score between 0.30 and 0.50. Scores below 0.30 can be considered a non-match. These score ranges can be used as initial grading. The range is set based on the need for multi-dimensional review scores to balance candidate recall and false positive control. Electronic devices can be calibrated based on manual review samples from the most recent 30 to 180 days. The 30-day samples are suitable for short videos and trending marketing materials, the 90-day samples are suitable for regular graphic and text advertising materials, and the 180-day samples are suitable for copyright assets with longer update cycles, such as fonts, templates, and background images. When the false positive rate of a certain type of material increases, the entry threshold of the strong matching range can be increased. When the false negative rate increases, the entry threshold of the candidate review range can be decreased, and the values ​​before adjustment, the values ​​after adjustment, the sample range, and the effective time are recorded. During the structural review process, the electronic device compares the object node positions, object arrangement, layout hierarchy, relative positions of images and text, and temporal relationships of video clips in the material relationship diagram with the candidate copyright assets. If the main image area, title text area, brand logo area, and auxiliary description area in the marketing material to be verified are close to the candidate template in relative position, even if the specific text or images have been replaced, it can be recorded as structurally similar. For video materials, the structural review can compare the shot order, subtitle appearance order, and scene transition rhythm. If the structural review level is higher than the content review level, the electronic device records it as a structurally dependent candidate. If the content review level is higher than the structural review level, it is recorded as a content-dependent candidate. If both the content review and structural review are in the strong matching range, it is recorded as a key review candidate. During the style review process, electronic devices read font samples, color relationships, decorative graphic styles, image processing methods, and layout styles for comparison. Font sample review can determine whether they are similar to fonts in candidate font libraries or historical materials based on font outlines, stroke thickness, character width-to-height ratios, and letter spacing. Color relationship review can determine whether they are similar to candidate materials based on the combination of primary color, secondary color, text color, and background color. Layout style review can be judged by combining white space ratios, alignment, hierarchy, and visual focus. If the marketing materials to be verified replace images and text with different content but retain the font, color scheme, and layout structure of the candidate template, the style review results can serve as supplementary evidence in addition to content review. During the audiovisual relationship verification process, the electronic device compares the time correspondence between video clips, subtitle content, and their appearance time with the audio clips in materials containing both video and audio with the candidate copyright assets. If the shot transition points and audio beat points in the candidate video template have a stable correspondence, and the marketing materials to be verified maintain the same correspondence in the same or similar time periods, it is recorded as having a similar audiovisual relationship. If the marketing materials to be verified change the speed or pitch of the background music, the electronic device can perform a rollback verification within a speed range of 0.8x to 1.25x. This range covers the mild acceleration and deceleration processing commonly seen on short video platforms. Beyond this range, the relationship between the audio beat and the video editing usually changes significantly, thus reducing the weight of the audiovisual relationship. For pitch-changing scenarios, verification can be performed within a range of two semitones above and below the pitch. This range covers common mild pitch-changing processing while avoiding the possibility of different music clips being mistakenly recalled due to an overly wide range. Candidate copyright asset retrieval can employ a combination feature-priority retrieval method, an asset type-based database retrieval method, or a historical evidence-guided retrieval method. The combination feature-priority retrieval method is suitable for scenarios requiring the discovery of similar overall expressions, where electronic devices prioritize the use of image-text combination features, layout combination features, and audiovisual combination features for candidate recall. The asset type-based database retrieval method is suitable for scenarios with a large copyright asset database and clearly defined asset types, where electronic devices search separately in image databases, font databases, music databases, template databases, and historical material databases, and then merge the candidate queues. The historical evidence-guided retrieval method is suitable for long-running platforms, where electronic devices adjust the candidate retrieval priority based on the characteristics of high-frequency false alarms, false alarms, and confirmed risky materials in historical review records. Regarding anomaly handling, if the copyright asset library is temporarily inaccessible, the electronic device prioritizes using the most recent local index snapshot for candidate retrieval and marks the candidate source as cached data. If a certain asset library is missing, such as the music library not being accessed, the electronic device skips the corresponding audio candidate retrieval and retains the audio ownership verification mark in subsequent authorization matching. If the number of retrieval candidates exceeds the upper limit, the electronic device sorts them according to metadata consistency, similarity of combined features, historical hit frequency, and asset type priority, retaining high-priority candidates for review. If all candidates are below the weak matching range, the electronic device generates an empty candidate review record and writes the overall characteristics of the marketing material to be verified into the history record for subsequent self-similar retrieval. The candidate copyright asset list and multi-dimensional review record formed by the above processing can serve as the data basis for subsequent authorization status matching.

[0024] Specifically, such as Figure 6 As shown: When performing authorization status matching and risk verification record generation, the electronic device reads the candidate copyright asset list, multi-dimensional review records, and candidate asset numbers, and calls the corresponding authorization status record based on the candidate asset number. The authorization status record can come from commercial material platform interfaces, enterprise contract management systems, internal material management systems, font authorization lists, music authorization lists, or manually entered records. The authorization status record includes at least the following fields: asset number, rights holder, authorizing entity, authorized entity, authorization start time, authorization end time, authorization region, authorization platform, authorization scenario, usage limit, secondary creation license status, commercial license status, cross-border distribution license status, authorization change time, and authorization certificate number. The authorization status can include unauthorized, limited authorization, full authorization, authorization suspended, authorization revoked, authorization expired, and pending confirmation status. When the electronic device calls the authorization status record, it prioritizes the authorization record directly corresponding to the candidate copyright asset number. If there is no directly corresponding record, it associates the asset with the asset source platform, asset name, authorization certificate number, or historical verification records to improve the matching stability between candidate assets and authorization records. When performing time matching, the electronic device compares the start and end times of the marketing materials to be verified with the authorization start and end times of the candidate copyright assets. If the delivery time falls entirely within the authorized time range, the time dimension is recorded as "coverage". If the delivery start time is earlier than the authorized start time, or the delivery end time is later than the authorized end time, the time dimension is recorded as "time out of bounds". If the delivery time field is missing, the time dimension is recorded as "time pending confirmation". For cross-time zone delivery scenarios, the electronic device uses the platform's unified timestamp for matching, while also retaining the local time of the submitting end to avoid errors in determining the authorization expiration time due to time zone conversion. For long-term delivery materials, the electronic device can generate time coverage records on a daily basis. If the authorized time only covers part of the delivery period, it is recorded as "partial time out of bounds". The above time matching method is based on the start and end times in the authorization file and the start and end times in the delivery task, and is a field range comparison, which does not rely on subjective judgment. When performing regional matching, the electronic device compares the target region of the marketing material to be verified with the authorized region in a tiered manner. The authorized region can include types such as global, country, province / city, platform region, or specific channel range. If the target region includes multiple regions, the electronic device compares them one by one and generates a regional coverage list. If the authorized region is China, but the target region is another country, the electronic device records it as incomplete regional coverage. If the authorized region is nationwide and the target region is within a province / city, the electronic device records it as regional coverage. If the authorized region field is missing but the authorization certificate number exists, the electronic device marks the region status as pending confirmation and does not directly record it as coverage. The regional matching priority can be judged in the following order: global authorization, country authorization, province / city authorization, platform region authorization, and unspecified authorization. When there are multiple authorized region records for the same candidate copyright asset, the authorization record with a more specific regional scope and valid certificate status is given priority to avoid the broad authorization field from obscuring specific restrictions. When matching platforms, the electronic device reads the target platform for the marketing materials to be verified and matches it with the authorized platform in the authorization status record. Authorized platforms can include e-commerce platforms, short video platforms, social media platforms, offline printing, outdoor advertising, live streaming platforms, or corporate websites. If the authorization record only allows e-commerce detail pages to be displayed, but the marketing materials to be verified are used for short video advertising, the electronic device records it as platform overreach. If the authorization record allows commercial placement on all platforms, the electronic device records it as platform overreach. For cases where platform names have aliases, version differences, or interface field differences, the electronic device can call the platform name mapping table for standardization. For example, mapping the international version, domestic version, merchant backend name, and placement system name of the same platform to a unified platform identifier. If the mapping fails, it is recorded as platform pending confirmation and manual review is triggered. When performing usage matching, the electronic device compares the usage field of the marketing material to be verified with the permitted usage in the authorization status record. Usage includes at least the following types: direct use, cropped use, color-changing use, text overlay, secondary adaptation, template application, background music editing, commercial placement, and cross-border placement. If the candidate copyright asset allows direct use but not secondary adaptation, and the multi-dimensional verification record shows that the marketing material to be verified has cropped, adjusted the color scheme, overlaid the text, or rearranged the template, the electronic device records it as secondary creation pending confirmation or usage exceeding the scope. If the candidate music asset allows use within short video platforms but not offline commercial activities, and the placement information includes offline materials or on-site playback at events, the electronic device records it as usage scenario not covered. Usage matching does not directly draw legal conclusions but generates verifiable technical verification records to indicate whether there are inconsistencies between similar candidate assets and the authorization field. When generating risk verification records, electronic devices comprehensively review records from multiple dimensions, as well as the matching results of time, region, platform, and usage method. Risk verification records must include at least the following fields: task number, candidate copyright asset number, review dimension, review level, authorization matching dimension, matching status, risk source location, manual review identifier, and generation time. Risk levels can be categorized as no abnormal risks found, low-risk (for review), medium-risk, high-risk, and risks pending resolution. If a candidate copyright asset achieves a strong match in at least two review dimensions, and at least one dimension—authorization time, authorization region, authorization platform, or usage method—is out of bounds, the electronic device generates a risk verification record. High-risk or pending risk records; if a candidate copyright asset achieves a strong match but all authorization dimensions are covered, the electronic device generates an authorization coverage record for future reference; if a candidate copyright asset achieves a medium match but authorization fields are missing, the electronic device generates a medium-risk record and triggers manual review; if a candidate copyright asset only achieves a weak match and no abnormalities are found in the authorization status, the electronic device generates a low-risk record for future reference; if no candidate copyright asset is found, the electronic device generates a risk record with no abnormalities and completes task archiving; the above risk levels are jointly formed by the multi-dimensional review level and the authorization field matching status, avoiding the direct generation of risk conclusions based solely on the similarity of materials; Authorization status matching can be performed using rule-based matching, status replay, or manual review and takeover. Rule-based matching is suitable for commercial material platforms with clear authorization field structures, where electronic devices compare each item according to time, region, platform, and usage method. Status replay is suitable for copyright assets with frequently changing authorization status, where electronic devices read the authorization status corresponding to the delivery time based on the authorization change time, such as authorization suspension, authorization resumption, authorization expiration, or authorization scope expansion. Manual review and takeover is suitable for materials with incomplete authorization certificates, inconsistent authorization entities, complex authorization chains, or unclear cross-border delivery restrictions, where electronic devices generate a risk verification record to be confirmed and submit candidate asset information, multi-dimensional review records, authorization matching process, and abnormal fields to the manual review interface. Regarding anomaly handling, if the authorization status record cannot be accessed, the electronic device marks the authorization status as pending confirmation and retains the candidate copyright asset and multi-dimensional review record in the risk verification record. If multiple authorization records exist for the same candidate copyright asset, the electronic device sorts them in the order of authorization time coverage, regional coverage, platform coverage, usage method coverage, and certificate validity, and selects the authorization record with the highest coverage and valid certificate for matching, while retaining other authorization records for future reference. If the authorization start time and authorization end time formats are inconsistent, the electronic device first converts them to a unified time format, and if the conversion fails, it proceeds to manual review. If missing delivery information prevents matching from being completed, the electronic device does not output a record of no abnormal risk found, but instead generates a record of missing fields. If the historical manual review results are inconsistent with the current matching results, the electronic device writes the difference fields into the review feedback record for subsequent updates to the authorization field mapping table and risk level determination rules.

[0025] Through the above continuous processing, electronic devices can convert marketing materials to be verified from raw files into hierarchical parsing objects, material relationship diagrams, combined feature units, candidate copyright asset review records, and authorization matching records. This processing chain enables the relationship between images, text, fonts, layouts, videos, subtitles, their appearance times, and audio clips to participate in copyright verification, and makes similar review content correspond to the placement time, placement area, placement platform, and usage method, thereby improving the stability of candidate recall, the consistency of review, the reliability of authorization matching, and the traceability of risk records.

[0026] Example 2: Based on Example 1, the specific application process of a multimodal marketing material copyright verification method is further explained: In a specific application scenario, the marketing materials to be verified are a set of new product marketing materials that an e-commerce platform plans to launch. This set of materials includes a vertical cover poster, a 20-second short video, video subtitles, background music, and corresponding placement metadata. This set of materials is intended for use in short video platform advertising and product detail page display, with the target region being China and the launch period being from June 1, 2026 to June 30, 2026. The usage methods include short video advertising, product page display, and secondary editing. The electronic device can be either the e-commerce platform's content review server or the brand's material management server, used to read the material files to be verified, material metadata, copyright asset library, authorization status records, and historical manual review records, and generate corresponding risk verification records. After receiving the marketing materials, the electronic device uniformly registers the material files and metadata. For vertical cover posters, the electronic device reads information such as image file format, file size, resolution, and color space. For 20-second short videos, the electronic device reads information such as video file format, frame rate, screen size, audio sampling rate, and subtitle file status. For metadata, the electronic device reads fields such as uploader identifier, material name, target platform, delivery area, delivery start time, delivery end time, usage method, material source description, authorization certificate number, and business scenario identifier. The electronic device establishes a task index based on the receiving time, uploader identifier, and file content summary value, so that cover posters, video files, subtitle files, and background music clips in the same set of materials can maintain a corresponding relationship under the same verification task. In this scenario, the vertical cover poster has a resolution of 1080 pixels by 1920 pixels, which is a common specification for vertical marketing materials on mobile devices and meets the clarity requirements for image region recognition, text region extraction, and layout structure recognition. The 20-second short video falls within the default whole-segment verification range of 1 second to 300 seconds and does not need to be split into multiple independent tasks as with long video tasks. The basis for setting this time range is that videos shorter than 1 second are difficult to form a stable correspondence between images, subtitles, and audio, while videos longer than 300 seconds usually contain multiple creative scenes or multiple advertising segments, making it more suitable to perform segmented verification first. The background music sampling rate is 44.1kHz, which is a common sampling rate range in music materials and video platform audio encoding. Electronic devices can use it as a data source for audio segment recognition and rhythm verification. If there are missing fields in the same material package, such as the absence of a secondary editing authorization certificate, the electronic device will not directly terminate the verification but will mark the field as pending supplementation in the task record and make this status participate in subsequent authorization matching. After receiving and verifying the materials, the electronic device performs layered analysis on the materials. For cover posters, the electronic device identifies the background area, the main product image area, the brand logo area, the promotional text area, the decorative graphic area, and the layout structure. For short videos, the electronic device reads key shots, camera transition positions, subtitle content and appearance time, audio segments, and screen size. For subtitles directly superimposed on the video screen, the electronic device extracts the subtitle content through on-screen text recognition and determines the subtitle appearance time by combining the duration of text appearance in adjacent frames. For background music, the electronic device determines audio segments based on silence boundaries, stable beat intervals, and video segment boundaries, enabling subsequent determination of whether there is a time correspondence between scene transitions and music beats. During image region recognition, the electronic device sets the initial threshold for the effective confidence of the object region to be above 0.75. This value is based on the trade-off between the false positive and false negative rates in historical manually reviewed samples, which can reduce significant false positives while retaining the main image, brand logo, and decorative elements. For high-definition cover posters in this scenario, since their resolution reaches 1080 pixels by 1920 pixels and the text edges are clear, the electronic device can increase the effective confidence of the object region to 0.80 to reduce the number of misidentified areas entering subsequent reviews. For video clips with significant motion blur, the electronic device can reduce the effective confidence of the object region to 0.70 and mark the corresponding objects as low-confidence objects, allowing them to participate in relationship establishment but reducing their review priority. Thus, even if there are fast transitions, partial occlusions, or semi-transparent decorations in the short video, potentially useful material objects will not be directly discarded due to insufficient confidence in a single recognition. During text and font parsing, the electronic device identifies the main title, promotional copy, product description text, and video subtitles in the cover poster, and extracts the corresponding font samples, font size range, character outline, stroke thickness, character spacing, and text color. If the text recognition confidence score of a certain subtitle area is lower than 0.80, the electronic device enters the character sampling backtracking process, and does not directly use the low-confidence text content, but retains the character outline, character height, stroke width, and text color. This value is higher than the effective confidence score of the image object because the text recognition result will simultaneously affect the copy verification, subtitle correspondence, and font sample grouping. If the low-confidence text content is used directly, it is easy to cause false positives or false negatives in the candidate assets. During the layout analysis process, the electronic device reads the relative positions of the main product image, brand logo, main title, price area, and decorative graphics in the cover poster. For mobile posters with a short side between 720 and 1080 pixels, the fixed grid side length can be selected from 32 to 64 pixels, which can balance the positioning accuracy of text blocks and the integrity of the layout structure. Since the short side of this cover poster is 1080 pixels, the electronic device can use a 64-pixel grid to initially divide the layout area, and then combine it with the object bounding box to record the vertical, horizontal, overlapping, adjacent, centered, and aligned relationships between the main image, title text, and brand logo. If the material is adjusted to a low-resolution version with a width of 720 pixels, the electronic device can proportionally reduce the grid side length so that the number of grids can still express the main text blocks, image blocks, and brand logo area. For a 20-second short video, the electronic device divides the video into segments based on the camera transition position and the boundaries of the subtitle paragraphs. Since the total video length does not exceed 30 seconds, the video segment unit can be selected from 3 to 5 seconds. This range can usually cover an advertising shot or a subtitle expression segment, which can avoid the semantic incompleteness caused by the segment being too short, and also avoid the mixing of multiple expression scenarios caused by the segment being too long. For example, the video can be divided into an opening product display segment, a function explanation segment, a usage scenario segment, and a promotional ending segment. If a subtitle spans two adjacent shots, the electronic device records the subtitle content into the time interval of the adjacent segment based on the duration of the subtitle's appearance, for subsequent use in establishing the time relationship between the video frame and the subtitle. After the layered analysis is completed, the electronic device establishes a material relationship graph based on the relationships between image regions, text regions, font samples, layout structure, video clips, subtitle content, appearance time, and audio clips. In this relationship graph, the main product image, promotional copy, brand logo, subtitle content, background music, and layout structure are respectively treated as object nodes; the correspondence between image and text, the correspondence between text and font, the correspondence between text and layout, the time relationship between video frame and subtitle, and the time relationship between video frame and audio are respectively treated as relationship edges. Through this processing, the product image and promotional copy in the cover poster are no longer compared as two unrelated features, but participate in the candidate retrieval as material objects with thematic and positional correspondence; the subtitles, shot transitions, and background music beats in the short video are also no longer processed separately, but participate in the review as material objects with time correspondence. When establishing the correspondence between text and images, the electronic device reads the promotional text content and the main product image content, and determines whether there is a thematic correspondence between the two. For example, if the promotional copy contains a new product name, functional description, or brand phrase, and the text area is close to the main product image area, the electronic device establishes a candidate correspondence. The proximity rule compares the distance from the center point of the text area to the center point of the image area with the length of the material's diagonal. When the proportion of this distance to the length of the material's diagonal does not exceed 0.35, a candidate correspondence is established first. This proportion can be adjusted between 0.25 and 0.45, where 0.25 is suitable for posters with the main body centered and the text arranged around the main image, and 0.45 is suitable for left-right or top-bottom layouts. The basis for setting this range is that the main title, main visual, and brand logo in advertising posters are usually located in the main expression area, which can reduce the situation where peripheral auxiliary text is incorrectly associated with the main visual area. When establishing the relationship between video, subtitles, and audio, the electronic device reads the start and end times of the video segment, the appearance time of the subtitle content, and the stable beat interval of the audio segment according to a unified timeline. If the overlap ratio between the appearance time of the subtitle and the time of a certain video segment reaches 0.60 or higher, the electronic device establishes a time correspondence between the subtitle content and the corresponding video segment. The basis for setting this ratio is that short video subtitles often appear earlier or disappear later than the on-screen actions, and an overlap ratio of 0.60 or higher can accommodate the common situation where subtitles cross the shot boundary. If the video belongs to a strong subtitle synchronization scenario, such as subtitles corresponding word by word to a product function demonstration, the overlap ratio can be increased to 0.70 or higher. If the stable beat interval of the audio segment is adjacent to the shot transition time, and the adjacent time difference does not exceed 0.5 seconds, the electronic device establishes an audiovisual correspondence. 0.5 seconds corresponds to a time range of approximately 15 frames per second for a 30-frame-per-second video and approximately 30 frames per second for a 60-frame-per-second video, which can cover the normal editing errors that exist between the shot transition and the beat point in short video editing. After the material relationship diagram is established, the electronic device extracts combined feature units. For cover posters, it can extract image-text combination feature units formed by the main product image and promotional copy, font combination feature units formed by the promotional copy and font samples, and layout combination feature units formed by the promotional copy and layout structure. For short videos, it can extract subtitle combination feature units formed by video clips, subtitle content, and appearance time, and audiovisual combination feature units formed by video clips and audio clips. If the main product image, main title, font samples, and layout structure in the cover poster together constitute the main visual expression, the electronic device can also combine the above multiple object nodes and multiple relationship edges to form the main visual combination feature unit. The above combined feature units are used to record the association information that a single material object cannot express on its own, so that subsequent searches can discover candidate assets with similar overall expressions but individual material objects that have been replaced or modified. Subsequently, the electronic device retrieves candidate copyright assets from the copyright asset library based on the combined feature units and the overall characteristics of the material. The copyright asset library may include the brand's own material library, the library of historically released materials, the commercial image licensing library, the commercial font licensing library, the commercial music licensing library, the video template library, the design template library, and the third-party commercial material licensing library. When a copyright asset is added to the library, an asset feature record is generated according to the same or corresponding hierarchical parsing rules as the marketing material to be verified. The asset feature record includes at least the following fields: image area, text area, font sample, layout structure, video clip, subtitle content and appearance time, audio clip, material relationship diagram, and combined feature units. In this way, the material to be verified and the assets in the copyright asset library can be compared based on the same type of data object. In this scenario, the electronic device can first map the cover poster to the image library, template library, font library, and historical poster library based on the material type, and map the short video to the video template library, music library, subtitle template library, and historical short video library. If the material source description records the material number of a commercial template platform, the electronic device will prioritize adding the corresponding asset from that platform to the high-priority candidate queue. If no direct source is recorded, candidate retrieval will be performed based on the image-text combination feature unit, layout combination feature unit, and audiovisual combination feature unit. For a copyright asset library of 100,000 to 10 million items, the number of candidates can be selected from 50 to 200, which can achieve a balance between candidate coverage and verification computation load. When the historical false negative rate increases, the number of candidates can be increased; when the false positive rate or single task processing time increases, the number of low-priority candidates can be reduced. After candidate asset retrieval, the electronic device performs multi-dimensional verification of the candidate copyright assets. Content verification compares the similarity of promotional copy, product images, subtitle content, and background music clips with the candidate assets. Structural verification compares the relative positions of the main image area, title text area, brand logo area, and auxiliary description area with the candidate template. Style verification compares the similarity of font samples, color relationships, decorative graphic styles, and layout styles with the candidate assets. Audiovisual relationship verification compares the stable and similar correspondence between shot transition points, subtitle appearance times, and audio beats. Each verification dimension can output a normalized verification score from 0 to 1. A strong match range can be selected as 0.80 or above, a medium match range can be selected as 0.50 to 0.80, and a weak match range can be selected as 0.30 to 0.50. Scores below 0.30 can be used as a non-match reference. The above score range is based on the fact that multi-dimensional verification needs to take into account both candidate recall and false positive control, and can be calibrated based on manual verification samples from the most recent 30 to 180 days. For example, if the electronic device finds during the review that the main product image in the cover poster to be verified only matches the candidate image asset in a moderate way, but the main title position, font sample, main color scheme, and decorative graphic arrangement are all in a strong matching range with a certain historical template asset, then the electronic device records this candidate as a structurally dependent candidate or a candidate for key review. Another example is if the background music in the short video to be verified is adjusted to 1.1x speed; the electronic device can perform a rollback review within the speed range of 0.8x to 1.25x, covering the slight acceleration and deceleration commonly seen on short video platforms. If the background music is slightly pitch-shifted, the electronic device can review within a range of two semitones above and below to avoid missing music candidates due to slight pitch shifts. The above review results do not directly form a legal judgment, but rather serve as the data basis for generating authorization matching and risk verification records. After the candidate copyright assets are determined, the electronic device accesses the corresponding authorization status record. The authorization status record includes at least the following fields: asset number, rights holder, licensor, authorized entity, authorization start time, authorization end time, authorization region, authorization platform, authorization scenario, secondary creation license status, commercial license status, cross-border distribution license status, authorization change time, and authorization certificate number. The electronic device prioritizes the authorization record that directly corresponds to the candidate copyright asset number. If no directly corresponding record exists, it associates the record based on the asset source platform, asset name, authorization certificate number, or historical verification records. If there are multiple authorization records for the same candidate asset, the electronic device sorts them in the order of authorization time coverage, region coverage, platform coverage, usage method coverage, and certificate validity, and selects the authorization record with the highest coverage and valid certificate for matching, while retaining other records for future reference. The electronic device compares the launch time of the marketing materials to be verified with the authorization time of the candidate copyright assets. If the launch period falls entirely between the authorization start time and authorization end time, the time dimension is recorded as overlay. If the launch end time is later than the authorization end time, it is recorded as time out of bounds. If the launch time field is missing, it is recorded as time pending confirmation. For cross-time zone launches, the electronic device uses the platform's unified timestamp for matching and retains the local time of the submitting end to avoid misjudgment of authorization expiration time due to time zone conversion. In this scenario, the launch period is from June 1, 2026 to June 30, 2026. If a font authorization record shows an authorization expiration date of June 15, 2026, the electronic device can generate a time overlay record on a daily basis and record the period from June 16, 2026 to June 30, 2026 as partial time out of bounds. The electronic device will also match the delivery area, target delivery platform, and usage method with the authorization record; if the authorized region of the candidate image asset is nationwide, and the delivery area of ​​the material to be verified is within the province or city of China, the region dimension will be recorded as coverage; if the authorized background music of the candidate is only allowed to be used within the short video platform, while the material to be verified is also used for offline material display or on-site playback at the event, the usage method will be recorded as usage scenario not covered; if the candidate template allows the product details page to be displayed but does not allow the short video ad delivery, the platform dimension will be recorded as platform overreach; for cases where the platform name has aliases or interface field differences, the electronic device can call the platform name mapping table to map the domestic version, international version, merchant backend name, and delivery system name of the same platform to a unified platform identifier. If the mapping fails, it will be recorded as platform pending confirmation and trigger manual review; Electronic devices generate risk verification records by integrating multi-dimensional review records and matching results based on time, region, platform, and usage method. These records include at least the task number, candidate copyright asset number, review dimension, review level, authorization matching dimension, matching status, risk source location, manual review identifier, and generation time. If a candidate template asset achieves a strong match in both structure and style reviews, and the template authorization only covers product detail page display and not short video ad placement, the electronic device generates a high-risk or pending-disposal risk record, pinpointing the risk source to the main visual area of ​​the cover poster and the corresponding segment in the short video. If a candidate font asset achieves a strong match but the authorization time and region both cover the current placement task, the electronic device generates an authorization coverage record for future reference. If a candidate quota achieves a medium match and the authorization field is missing, the electronic device generates a medium-risk record and triggers manual review. If a candidate asset only achieves a weak match and no abnormalities are found in the authorization status, the electronic device generates a low-risk record for future reference. If no candidate copyright asset is found, the electronic device generates a risk record indicating no abnormalities and completes task archiving. In this specific application, the marketing materials to be verified form a continuous data processing chain, from file reception, hierarchical parsing, relationship establishment, candidate retrieval, multi-dimensional review to authorization matching. This processing chain can include the relationship between cover images, copy, fonts, layouts, video clips, subtitle content, appearance time, and background music clips in the verification scope, and establish a corresponding relationship between the review results and the placement time, placement area, placement platform, and usage method. As a result, electronic devices can identify materials with low similarity in a single image or a single sentence of copy but similar overall expression, and can also distinguish between similar materials with overlapping authorization and similar materials with out-of-bounds authorization, thereby improving the stability of candidate recall, consistency of review, reliability of authorization matching, and traceability of risk records.

[0027] It should be noted that this invention can be deployed on the device itself to realize embedded applications, or it can run on a PC or other terminal with a user interface, thereby meeting various hardware environments and usage requirements.

[0028] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above embodiments can be implemented in whole or in part by a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions of the embodiments of this application are implemented in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted wirelessly or wiredly from one website, computer, server, or data center to another website, computer, server, or data center. Wired methods include optical fiber, twisted pair, coaxial cable, etc. Wireless methods include infrared, microwave, etc. Available media include any available media that can be accessed by a computer or data storage devices such as servers and data centers that contain one or more sets of available media. Available media can be magnetic media (floppy disks, hard disks, magnetic tapes), optical media (DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0029] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0030] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for verifying the copyright of multimodal marketing materials, characterized in that, include: S1: Receive marketing materials to be verified and corresponding metadata, verify the material file format, placement information and authorization verification fields, and establish a verification task record; S2: Perform layered analysis on marketing materials to be verified, identifying image regions, text regions, font samples, layout structure, video clips, subtitle content and appearance time, and audio clips; select full, lightweight, or abnormal rollback analysis methods based on material type and list of parseable files; perform region segmentation, confidence marking, and layout positioning on images or posters; extract font shape, font size, and color from text and form font samples; divide videos into segments according to shot transitions, subtitle segments, or time windows; write the object type, location coordinates, time interval, source file, and confidence level of each analyzed object into the layered analysis record; read the material type, file format verification results, and list of parseable files from the verification task record; perform full analysis for pre-release review tasks; perform lightweight analysis for creation-end warning tasks; when the video has no audio, the image has no text, or the subtitle file is corrupted, perform abnormal rollback analysis and process it according to the parseable image, text, subtitle, or audio modality; S3: Based on the hierarchical analysis results, establish a material relationship diagram and extract combined feature units between images, text, fonts, layouts, videos, and audios; set the parsed objects in the hierarchical analysis records as object nodes; set semantic correspondence, positional correspondence, time synchronization, style matching, and source correspondence as relationship edges; establish relationships between object nodes according to spatial proximity, theme keywords, font size level, color difference value, and time overlap; set anomaly markers for low-confidence objects, misaligned subtitle timelines, or missing audio; extract combined feature units from image-text correspondence, font correspondence, layout correspondence, subtitle correspondence, and audiovisual correspondence. The material relationship diagram consists of object nodes and relationship edges. When establishing the correspondence between text and images, the electronic device reads the text content of the text area and the image objects of the image area, and establishes candidate correspondences based on the thematic consistency between the text content and the image objects. Candidate correspondences are determined jointly by position proximity rules, content keyword rules, and region hierarchy rules. The position proximity rule compares the distance from the center point of the text region to the center point of the image region with the length of the material's diagonal. When establishing font, layout, and style relationships, the electronic device combines the font samples, text colors, font size ranges, row and column arrangements, and layout structures corresponding to the same text region. When establishing video, subtitle, and audio relationships, the electronic device reads the start and end times of video clips, the appearance time of subtitle content, and the start and end times of audio clips according to a unified timeline. If the overlap ratio between the appearance time of subtitles and the time of video clips reaches 0.60 or more, the electronic device establishes a time correspondence between the subtitle content and the corresponding video clip. If the stable tempo interval of an audio clip is adjacent to the shot transition time, and the adjacent time difference does not exceed 0.5 seconds, the electronic device establishes an audiovisual correspondence between the audio clip and the video clip. S4: Based on the combined feature units and the overall characteristics of the material, retrieve candidate copyright assets in the copyright asset database, and review the content, structure, style, and audiovisual relationship of the candidate copyright assets; determine the asset database to be retrieved according to material type, distribution platform, brand logo, material source, and asset tag; generate a candidate queue based on the combined feature units, the overall characteristics of the material, and historical review records; the overall characteristics of the material are formed by the electronic device reading the previously formed combined feature units and combining them with image areas, text areas, font samples, layout structure, video clips, subtitle content and appearance time, and audio clips during the candidate copyright asset retrieval and review process. The text, images, subtitles, audio, layout, font, color relationships, and audiovisual timing relationships of candidate copyright assets are reviewed separately; a review flag is set when the asset database is inaccessible, a sub-database is missing, the number of candidates exceeds the upper limit, or the matching level is lower than the preset level; During the content review process, the electronic device compares the text content, image area content, subtitle content, and audio segment content in the marketing materials to be verified with the candidate copyright assets respectively; During the structural review process, the electronic device compares the object node positions, object arrangement, layout hierarchy, relative positions of text and images, and temporal relationships of video clips in the material relationship diagram with the candidate copyright assets; During the style review process, the electronic device reads and compares font samples, color relationships, decorative graphic styles, image processing methods, and layout styles; during the audiovisual relationship review process, the electronic device compares the time correspondence between video clips, subtitle content, and their appearance time and audio clips with the candidate copyright assets for materials containing video and audio. S5: Call the authorization status record of the candidate copyright assets, match the review content with the release time, release area, release platform and usage method, and generate risk verification record.

2. The method for verifying the copyright of multimodal marketing materials according to claim 1, characterized in that, S1 includes: Material files and metadata can be obtained through the material submission interface, batch import interface, pending review queue, or creator-side channel; Verify the file header, encoding format, resolution, frame rate, sampling rate, subtitle duration, compressed file directory, delivery field, and authorization field; Establish a task index based on the receiving time, uploader identifier, and file content summary value; Set flags for missing fields, unparseable files, inconsistent frame rates, audio that needs parsing, or abnormal credentials.

3. The method for verifying the copyright of multimodal marketing materials according to claim 2, characterized in that, A task index is built based on the receiving time, the uploader identifier, and the file content digest value, including: Standardize the receiving time into a unified timestamp; Associate the uploader identifier, file content summary value, and verification task number; Generate sub-file numbers for image files, video files, audio files, subtitle files, and description files within the same resource package; Write the sub-file number, material type, and source file information into the index record.

4. The method for verifying the copyright of multimodal marketing materials according to claim 1, characterized in that, A candidate queue is generated based on combined feature units, overall material features, and historical review records, including: Candidate assets are obtained according to asset types: images, fonts, music, templates, and historical materials. Set the assets whose source descriptions correspond to their asset tags as high-priority candidate assets; Supplement candidate assets based on the characteristics of text and image combination, layout combination, audiovisual combination, and overall material characteristics; Candidate assets are sorted according to historical hit frequency, historical false alarm records, and asset type priority.

5. The method for verifying the copyright of multimodal marketing materials according to claim 1, characterized in that, S5 includes: Retrieve authorization status records based on candidate asset number; When no direct corresponding record is obtained, the authorization record shall be associated with the asset source platform, asset name, authorization certificate number or historical verification record; Match the authorized time, authorized region, authorized platform, and permitted usage method with the corresponding delivery fields one by one; Set a review flag when the platform name is inconsistent, there are multiple authorization records, authorization status records are missing, or the delivery field is missing; Write the risk verification record according to the review level and matching status.

6. The method for verifying the copyright of multimodal marketing materials according to claim 5, characterized in that, Match the authorization time, authorization region, authorization platform, and permitted usage method with the corresponding delivery fields item by item, including: Compare the launch start time and launch end time with the authorized start time and authorized end time respectively within the time interval; The deployment areas and authorized regions are compared and classified. The target delivery platforms are standardized according to the platform name mapping table and then compared with the authorized platforms; Compare the usage of the material with the permitted usage in the authorization status record.

Images