Marketing video auditing method based on AI

Abstract

The invention provides an AI-based marketing video auditing method, and relates to the technical field of AI marketing video auditing, and the method comprises the steps: obtaining a multi-modal data original structure set, and extracting image semantic features, voice expression features, text semantic features and scene label information, and obtaining an image semantic feature set, a visual rhythm feature set, a voice expression feature set, a voice and picture synchronous association vector structure, a text semantic feature set and a subtitle semantic and image main body linkage relation graph. By constructing an image semantic feature set, a voice expression feature set, a text semantic feature set and a visual rhythm feature set and fusing the image semantic feature set, the voice expression feature set, the text semantic feature set and the visual rhythm feature set into a multi-modal content fusion feature tensor, unified modeling of an AI marketing video at visual, auditory and semantic levels can be realized; and subsequent microscopic consistency detection, compliance knowledge graph and emotion semantic conflict identification are effectively performed, so that full-link risk perception and accurate auditing of video contents are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of AI marketing video review, and more particularly to an AI-based marketing video review method. Background Art

[0002] With the development of artificial intelligence and short video platforms, AI-generated marketing videos have gradually become an important means of digital marketing. Compared with traditional advertising materials, AI marketing videos have the advantages of high generation efficiency, content diversity and precise semantic expression. They are widely used in e-commerce promotion, product recommendation and brand promotion scenarios. The current mainstream review strategies mostly rely on manual labeling, rule templates or machine learning models based on single-modal content for judgment. They cannot fully deal with the complex compliance, emotional orientation and subjective expression conflicts contained in multimodal content. Risk factors, therefore, need to build an intelligent review method based on multimodal feature understanding and behavioral semantic recognition to enhance the capabilities of AI marketing videos in automation, structured and explainable review.

[0003] However, existing AI video review methods have the following key flaws: First, there is a lack of a deep semantic fusion mechanism for multimodal content such as images, voice and text, which makes it impossible for the review system to accurately identify the linkage information between different modalities, which can easily lead to misjudgments or omissions. Second, there is a lack of systematic analysis of the consistency between emotional expression and marketing intentions. Especially in videos that express subjective emotions, it is easy to ignore potentially misleading expressions, resulting in content-oriented risks. These defects reduce the accuracy and interpretability of the review results in the actual review process, and also limit the platform's ability to regulate the compliance of marketing content. Summary of the Invention

[0004] The purpose of the present invention is to address the shortcomings of the existing technology and provide an AI-based marketing video review method. It aims to achieve deep fusion analysis of multimodal content by constructing an image semantic feature set, a speech expression feature set, a text semantic feature set, a visual rhythm feature set and a multimodal content fusion feature tensor, and further combine facial expression and speech emotion consistency detection, marketing intention recognition and emotional semantic conflict scoring model to achieve accurate identification of potentially misleading content in the video. Through the compliance knowledge graph path matching scoring mechanism and the structured violation causal path generation method, the logical reasoning ability and compliance recognition coverage of the review system are improved. Finally, the system constructs an AI marketing video quality grading assessment model based on the above content, realizes the integrated and closed-loop processing of review and quality assessment, and significantly improves the review effect of AI marketing videos in terms of compliance, accuracy and explainability.

[0005] To achieve the above object, the present invention provides the following technical solutions: The AI-based marketing video review method includes the following steps: Step S100: Obtain a set of original structures of multimodal data, extract an image semantic feature set, a visual rhythm feature set, a speech expression feature set, an audio-visual synchronization association vector structure, a text semantic feature set, a subtitle semantics and image subject linkage relationship diagram, a scene label information set and a scene relationship mapping diagram, and configure a multimodal content fusion feature tensor structure.

[0006] Step S200: Based on the feature set, the matching degree between character expression and voice emotion, lip shape and voice synchronization, action rationality, camera movement rationality, scene logic continuity and semantic emotion consistency are detected, and a micro-video content consistency abnormal feature set is configured.

[0007] Step S300: Configure a compliance knowledge graph dedicated to AI marketing videos, use multimodal content fusion feature tensors to perform semantic path matching, identify complex violation behavior paths, and generate a structured violation path list and preliminary compliance assessment results.

[0008] Step S400: Configure a subjective emotion tendency label set and a marketing intention label sequence, calculate an emotion semantic conflict score, identify misleading expression segments, and generate an emotion semantic conflict risk label and a structured conflict risk data list.

[0009] Step S500: Aggregate multiple types of violation identification results to form a violation risk feature matrix, track the triggering source of the violation behavior, and generate an explainable violation risk report and a structured modification suggestion list.

[0010] Step S600: Configure multi-dimensional scoring dimensions and comprehensive scoring functions, and output quality grade labels and quality assessment reports.

[0011] As a preferred solution of the present invention, step S100 is specifically as follows: Step S100.1: Obtain a set of original structures of multimodal data and extract image semantic features.

[0012] The system accepts AI marketing videos for review, wherein the AI ​​marketing videos include: image frame sequences, voice tracks, subtitle texts, and data channels for background scenes.

[0013] The AI ​​marketing video is input into the video structure decoding module, decoding processing is performed, time-synchronized image frame sequences, audio tracks and subtitle layers are extracted, and the relationships between frames are marked to form a set of original structures of multimodal data.

[0014] The image frame sequence is input into the image semantic feature extraction network. The image semantic feature extraction network uses a trained multi-scale convolutional neural network structure to analyze the facial information of the characters, the shape information of the products, the brand logo pattern, the primary and secondary layout of the images, and the dynamic relationship of the actions. It performs semantic encoding processing on each frame of the image to obtain a set of image semantic features.

[0015] It further performs motion analysis operations between image frame sequences, and uses the image frame sequence differential analysis algorithm, bidirectional optical flow estimation algorithm and key frame judgment algorithm to extract the lens motion characteristics, lens advancement rhythm characteristics and lens switching density characteristics between image frame sequences to form a visual rhythm feature set of the image frame sequence.

[0016] Step S100.2: Extract speech expression features based on the original structure set of multimodal data.

[0017] The audio track is input into the speech expression feature extraction network, which uses an emotion modeling network based on a deep acoustic modeling structure to extract speech rate, pitch, tone, voice volatility and tone type from the human voice and form a speech expression feature set.

[0018] The speech expression feature set is synchronously aligned with the visual rhythm feature set of the image frame sequence in the image frame sequence to form an audio-visual synchronization association vector structure.

[0019] Step S100.3: Extract text semantic features based on the original structure set of multimodal data.

[0020] Perform text detection and recognition operations on each frame image in the subtitle layer, call the video text recognition module to extract the subtitle content and advertising description, and output the timed subtitle text sequence corresponding to the frame sequence, and input the subtitle text sequence into the text semantic feature extraction network. The text semantic feature extraction network adopts a natural language understanding model based on contextual semantic modeling to identify promotional verbs, functional descriptive words, suggestive evaluation words and risk descriptions, and performs semantic embedding coding on each identified subtitle sentence, extracts the text semantic feature set, and further performs spatial relationship modeling between the subtitle position and the main object of the picture based on subtitle recognition, and constructs a linkage relationship diagram between subtitle semantics and image main body.

[0021] Step S100.4: Extract scene label information based on the original structure set of multimodal data, and configure the multimodal content fusion feature tensor.

[0022] The image frame sequence is input into the scene label extraction network. The scene label extraction network is a trained image classification neural network that performs frame-by-frame scene recognition operations on the image frame sequence, identifying: medical environment, school environment, home space, public passage or commercial store, and generates a corresponding scene label information set.

[0023] Further identify whether there are background brand logo patterns, competing product patterns, and the positional relationship between the characters and main objects in the scene in the scene image, and construct a scene relationship map.

[0024] The image semantic feature set, visual rhythm feature set, speech expression feature set, audio-visual synchronization association vector structure, text semantic feature set, subtitle semantics and image subject linkage relationship diagram, scene label information set and scene relationship mapping diagram are uniformly encoded and mapped into a unified multimodal feature space using a fusion embedding mechanism, and a complete multimodal content fusion feature tensor structure is constructed.

[0025] As a preferred solution of the present invention, step S200 is specifically as follows: Step S200.1: Detect the degree of matching between the facial expression and speech emotion of the character and the synchronization between the lip movements and speech pronunciation of the character based on the image semantic feature set and the speech expression feature set.

[0026] Step S200.2: Detecting the rationality of the physical movements of the person or object based on the image frame sequence, and detecting the rationality of the lens operation strategy based on the visual rhythm feature set.

[0027] The motion trajectory data of people and objects is extracted based on the image frame sequence difference algorithm. The optical flow estimation network and the object posture prediction network are used to calculate the continuity and physical rationality indicators of the motion trajectory, including speed change, direction change angle, and position offset. When the speed change exceeds 500 pixels / second, the direction change angle is greater than 90 degrees, or the position offset between two consecutive frames exceeds 100 pixels without a transition frame, it is marked as an unreasonable physical motion anomaly. Abnormal phenomena include floating, transients, and penetration.

[0028] The lens advancement rhythm features and lens switching density features in the visual rhythm feature set are obtained, and their conformity to the human eye's visual adaptation rules is analyzed. When the continuous frame segments with a lens switching interval of less than 200 milliseconds exceed 5 seconds, the lens advancement speed mutation rate exceeds 80%, or unnatural lens jitter lasts for more than 10 frames, it is marked as abnormal lens operation strategy. Abnormal lens operation strategy includes unnatural advancement, lens jumps without physical transitions, artificial handheld shaking, and non-smooth zooming.

[0029] Step S200.3: Detect the logical continuity of the background scene based on the scene label information set, detect the consistency of semantics and voice emotions based on the text semantic feature set and the voice expression feature set, and configure the micro video content consistency anomaly feature set.

[0030] A frame-sequence scene flow graph is constructed based on the scene label information set to analyze whether the continuous scene labels conform to the actual scene logic. When different logical scenes are switched continuously in a short period of time without any intermediate transition scenes, it is marked as a background scene logic break anomaly. The position continuity of the characters in different scenes is detected based on the scene relationship mapping graph. When it is detected that the position of the characters in different scenes changes instantly and there is a lack of reasonable movement process, it is marked as a character spatial position discontinuity anomaly.

[0031] The semantic type of the subtitle sentence is identified based on the text semantic feature set, and the corresponding voice tone type is identified based on the voice expression feature set. The consistency between the subtitle semantic type and the voice tone type is compared through the semantic tonality mapping table, and the consistency score is calculated. When the consistency score is lower than the threshold of 0.5, it is marked as an anomaly of inconsistency between semantics and voice emotion.

[0032] The anomalies such as inconsistency between facial expressions and voice emotions, asynchrony between lip movements and voice pronunciation, unreasonable physical movements, abnormal camera operation strategies, logical breaks in background scenes, discontinuous spatial positions of characters, and inconsistency between semantics and voice emotions are uniformly coded to generate a set of micro-video content consistency anomaly features, and the corresponding frame sequence positions and anomaly score values ​​are marked.

[0033] As a preferred solution of the present invention, step S300 is specifically as follows: Step S300.1. Configure a compliance knowledge graph dedicated to AI marketing videos, match the semantic paths of the compliance knowledge graph based on the multimodal content fusion feature tensor, and obtain a path matching score.

[0034] Step S300.2: Identify the composite violation path based on the path matching score.

[0035] Step S300.3: Generate a compliance assessment tag based on the violation path matching score value, and generate a structured violation path list and a preliminary compliance assessment result based on the compliance assessment tag.

[0036] The system classifies and structures the single violation paths and compound violation paths identified by the semantic path matching engine, and generates a compliance assessment label for each violation path. The system contains five compliance assessment labels, namely: Information on the compliance assessment label: 1. Violation types include: medical efficacy not clearly stated violation, visual and voice exaggeration violation, and competitor attack violation.

[0037] Information 2 of the compliance assessment label: the compliance review semantic node sequence corresponding to the violation path.

[0038] Information 3 of the compliance assessment label: Multimodal feature combination type triggered by the violation path, including: image violation features, text violation features, and voice violation features.

[0039] Information 4 of the compliance assessment label: the starting frame number and the ending frame number corresponding to the illegal video clip.

[0040] Information 5 of the compliance assessment label: Matching score value of the violation path.

[0041] Based on the matching score value and the severity of the violation type in the compliance assessment label, all violation paths and corresponding video clip information are organized into a structured violation path list. The preliminary compliance assessment results of the AI ​​marketing video are generated based on the number of violation paths, violation types and path matching score values.

[0042] As a preferred solution of the present invention, step S400 is specifically as follows: Step S400.1: Configure a subjective emotion tendency label set based on the speech expression feature set, and identify a marketing intention label sequence based on the text semantic feature set.

[0043] Step S400.2: Configure a conflict scoring function between the emotion tendency tag set and the marketing intention tag sequence, and obtain an emotion semantic conflict score value.

[0044] Step S400.3: Identify misleading expression segments based on the emotion-semantic conflict score, and generate an emotion-semantic conflict risk label and a structured conflict risk data list.

[0045] The emotional semantic conflict score value calculation is performed on the entire video frame. When the emotional semantic conflict score value of any frame , it is marked as a high-risk frame for potential misleading expression, and a window sliding detection method is used in the frame sequence timeline. If the number of high conflict score frames exceeds 45 frames continuously, the corresponding frame segment is identified as a potential misleading expression segment, and the start and end time information of the current frame segment is output.

[0046] For all segments identified as potentially misleading expressions, an emotional semantic conflict risk label is generated. The emotional semantic conflict risk label includes: the combination type of conflict between emotional tendency and marketing intention, the start and end frame numbers of the video segment, the maximum conflict score, the average semantic jump rate, the maximum emotional confidence fluctuation value, the conflict level, and whether it covers the main sentence expression.

[0047] As a preferred solution of the present invention, step S500 is specifically as follows: Step S500.1: Aggregate multiple types of violation identification results to form a violation risk feature matrix, and track the violation causal path and locate the violation trigger source.

[0048] Step S500.2: Mark high-risk frame segments according to the calculation results.

[0049] Step S500.3: Perform causal path tracing of violation behaviors and identification of trigger sources, generate structured violation modification suggestions and mark applicable areas, and output an explainable violation risk report and a structured modification suggestion list.

[0050] For all high-risk video clips, the violation causal path tracing algorithm is called to use the violation type, violation label, violation trigger mode type, violation description text and violation feature point frame index information marked in the violation risk feature matrix, combined with the compliance knowledge graph to perform path backtracking, extract the violation trigger chain and the starting causal point, and output the violation event causal path structure. The violation event causal path structure includes: violation type, first trigger frame number, violation source mode, associated violation content description text, violation feature point frame index, hit compliance knowledge graph clause node sequence and the start and end frame sequence range of the high-risk frame segment involved.

[0051] Based on the violation type, violation source mode, associated violation content description text and hit compliance knowledge graph clause node sequence extracted from the causal path structure of the violation event, a structured violation modification suggestion list is constructed. The suggestions include: recommended operation type, recommended content description, recommended applicable frame segment start and end frame sequence numbers, involved violation reason text, based on the compliance knowledge graph clause node number and rule content and expected violation risk score reduction range.

[0052] The structured violation modification suggestion list is output in JSON structure and table structure respectively.

[0053] Based on the violation risk feature matrix, the total violation risk score, and the structured violation modification suggestion list, an explainable violation risk report in a standard format is generated, including: a full-frame violation risk heat map of the video, showing the trend of frame-level violation score changes, a statistical chart of violation events classified by violation type and a pie chart of violation ratios, an index of the start and end frames of all high-risk clips, the violation types and violation labels involved, a visualization map of the causal path of violation events, a structured violation modification suggestion list, and the total violation risk score and platform release suggestions.

[0054] The report is output in PDF format and WebJSON structure respectively.

[0055] As a preferred solution of the present invention, step S600 is specifically as follows: Step S600.1: Configure the multi-dimensional scoring dimensions of the AI ​​marketing video quality grading evaluation model and configure the AI ​​marketing video quality comprehensive scoring function.

[0056] Configure an AI marketing video quality grading and evaluation model. The AI ​​marketing video quality grading and evaluation model performs multi-dimensional quality scoring based on the fusion of multi-source feature information input. The input multi-source feature information includes: multimodal content fusion feature tensor, micro-video content consistency anomaly feature set, emotional semantic conflict score, path matching score and platform communication feedback indicator set. The platform communication feedback indicator set includes four core feedback parameters: video completion rate, number of likes, comment interaction frequency and content forwarding number.

[0057] Five core scoring dimensions are constructed based on the input multi-source feature information, namely: video naturalness scoring item, visual coherence scoring item, emotional marketing consistency scoring item, violation risk scoring item and platform communication feedback scoring item. Each scoring item adopts a normalized scoring mechanism with a score range of [0,1], and a comprehensive scoring function is introduced to calculate the comprehensive score value.

[0058] Step S600.2: Generate a quality grade label based on the comprehensive scoring function result and output a quality assessment report.

[0059] The quality level of AI marketing videos is marked based on the comprehensive score. The classification criteria include: When , it is marked as high quality, when When , it is marked as medium quality, when , marked as low quality.

[0060] Generates a structured quality assessment report, which includes: video quality grade labels, detailed rating values ​​for each rating item, a heat map of video frame-level violation risks, and a summary of marketing sentiment and communication effect analysis.

[0061] The structured quality assessment report will be output in WebJSON structure and PDF format respectively.

[0062] For high-quality videos, a green release mark is automatically assigned, the system review depth is set to the minimum, and only risk spot checks are performed. For medium-quality videos, the system triggers a local enhanced review process, focusing on detecting risky frames and semantically deviated segments. For low-quality videos, the system outputs detailed quality optimization suggestions to the content publisher and temporarily suspends the push process until corrections are made and the score is re-scored.

[0063] Compared with the prior art, the present invention has the following beneficial effects: 1. By constructing an image semantic feature set, a visual rhythm feature set, a speech expression feature set, a text semantic feature set, a subtitle semantics and image subject linkage relationship diagram, a scene label information set and a scene relationship mapping diagram, and generating a unified multimodal content fusion feature tensor based on a fusion embedding mechanism, it is possible to achieve synchronous modeling and unified expression of the image, speech and text dimensions of AI marketing videos without relying on manual rule setting, thereby improving the semantic linkage capability and context consistency perception capability between multi-source data, and providing complete data support for subsequent behavior consistency judgment, semantic path matching and video quality scoring.

[0064] 2. By building a compliance knowledge graph path matching engine, a micro-video content consistency anomaly detection mechanism, and an emotional semantic conflict scoring mechanism, it can identify multiple types of high-risk violations, including: inconsistency between facial expressions and voice emotions, asynchrony between lip movements and voice pronunciation, abnormal camera advancement strategies, conflicts between marketing intentions and emotional expressions, etc., and generate a structured violation path list, a causal path structure of violation events, and a list of violation modification suggestions. It has traceable, explainable, and visual output capabilities, realizing a closed-loop link from violation identification to rectification suggestions, and improving the intelligence and precision of content review.

[0065] 3. By building an AI marketing video quality grading and assessment module, the system integrates the video naturalness score, visual coherence score, emotional and semantic consistency score, compliance path score and platform communication feedback indicators to form a quantitative total quality assessment score, and generates low-quality, medium-quality and high-quality grade labels based on this. Based on the grade results, the system can automatically call the review process of different depths or provide content optimization suggestions to the publisher, thereby realizing on-demand allocation of review process resources and adaptive adjustment of risk response strategies, improving review efficiency while ensuring content compliance and communication security. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A flowchart of the AI-based marketing video review method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0068] See also Figure 1 , Figure 1 A flowchart of an AI-based marketing video review method is provided for an embodiment of the present application.

[0069] In this embodiment, the AI-based marketing video review method may include steps S100, S200, S300, S400, S500, and S600; Step S100: Obtain a set of original structures of multimodal data, extract an image semantic feature set, a visual rhythm feature set, a speech expression feature set, an audio-visual synchronization association vector structure, a text semantic feature set, a subtitle semantics and image subject linkage relationship diagram, a scene label information set and a scene relationship mapping diagram, and configure a multimodal content fusion feature tensor structure.

[0070] Step S200: Based on the feature set, the matching degree between character expression and voice emotion, lip shape and voice synchronization, action rationality, camera movement rationality, scene logic continuity and semantic emotion consistency are detected, and a micro-video content consistency abnormal feature set is configured.

[0071] Step S300: Configure a compliance knowledge graph dedicated to AI marketing videos, use multimodal content fusion feature tensors to perform semantic path matching, identify complex violation behavior paths, and generate a structured violation path list and preliminary compliance assessment results.

[0072] Step S400: Configure a subjective emotion tendency label set and a marketing intention label sequence, calculate an emotion semantic conflict score, identify misleading expression segments, and generate an emotion semantic conflict risk label and a structured conflict risk data list.

[0073] Step S500: Aggregate multiple types of violation identification results to form a violation risk feature matrix, track the triggering source of the violation behavior, and generate an explainable violation risk report and a structured modification suggestion list.

[0074] Step S600: Configure multi-dimensional scoring dimensions and comprehensive scoring functions, and output quality grade labels and quality assessment reports.

[0075] In some specific implementations, the step S100 is specifically: Step S100.1: Obtain a set of original structures of multimodal data and extract image semantic features.

[0076] The system accepts AI marketing videos for review, wherein the AI ​​marketing videos include: image frame sequences, voice tracks, subtitle texts, and data channels for background scenes.

[0077] The AI ​​marketing video is input into the video structure decoding module, decoding processing is performed, time-synchronized image frame sequences, audio tracks and subtitle layers are extracted, and the relationships between frames are marked to form a set of original structures of multimodal data.

[0078] The image frame sequence is input into the image semantic feature extraction network. The image semantic feature extraction network uses a trained multi-scale convolutional neural network structure to analyze the facial information of the characters, the shape information of the products, the brand logo pattern, the primary and secondary layout of the images, and the dynamic relationship of the actions. It performs semantic encoding processing on each frame of the image to obtain a set of image semantic features.

[0079] It further performs motion analysis operations between image frame sequences, and uses the image frame sequence differential analysis algorithm, bidirectional optical flow estimation algorithm and key frame judgment algorithm to extract the lens motion characteristics, lens advancement rhythm characteristics and lens switching density characteristics between image frame sequences to form a visual rhythm feature set of the image frame sequence.

[0080] Step S100.2: Extract speech expression features based on the original structure set of multimodal data.

[0081] The audio track is input into the speech expression feature extraction network, which uses an emotion modeling network based on a deep acoustic modeling structure to extract speech rate, pitch, tone, voice volatility and tone type from the human voice and form a speech expression feature set.

[0082] The speech expression feature set is synchronously aligned with the visual rhythm feature set of the image frame sequence in the image frame sequence to form an audio-visual synchronization association vector structure.

[0083] Step S100.3: Extract text semantic features based on the original structure set of multimodal data.

[0084] Perform text detection and recognition operations on each frame image in the subtitle layer, call the video text recognition module to extract the subtitle content and advertising description, and output the timed subtitle text sequence corresponding to the frame sequence, and input the subtitle text sequence into the text semantic feature extraction network. The text semantic feature extraction network adopts a natural language understanding model based on contextual semantic modeling to identify promotional verbs, functional descriptive words, suggestive evaluation words and risk descriptions, and performs semantic embedding coding on each identified subtitle sentence, extracts the text semantic feature set, and further performs spatial relationship modeling between the subtitle position and the main object of the picture based on subtitle recognition, and constructs a linkage relationship diagram between subtitle semantics and image main body.

[0085] Step S100.4: Extract scene label information based on the original structure set of multimodal data, and configure the multimodal content fusion feature tensor.

[0086] The image frame sequence is input into the scene label extraction network. The scene label extraction network is a trained image classification neural network that performs frame-by-frame scene recognition operations on the image frame sequence, identifying: medical environment, school environment, home space, public passage or commercial store, and generates a corresponding scene label information set.

[0087] Further identify whether there are background brand logo patterns, competing product patterns, and the positional relationship between the characters and main objects in the scene in the scene image, and construct a scene relationship map.

[0088] The image semantic feature set, visual rhythm feature set, speech expression feature set, audio-visual synchronization association vector structure, text semantic feature set, subtitle semantics and image subject linkage relationship diagram, scene label information set and scene relationship mapping diagram are uniformly encoded and mapped into a unified multimodal feature space using a fusion embedding mechanism, and a complete multimodal content fusion feature tensor structure is constructed.

[0089] In some specific implementations, the step S200 is specifically: Step S200.1: Detect the degree of matching between the facial expression and speech emotion of the character and the synchronization between the lip movements and speech pronunciation of the character based on the image semantic feature set and the speech expression feature set.

[0090] The facial information of the characters in the image semantic feature set is input into the multi-label convolutional emotion recognition network to identify the facial expression categories of the characters, including joy, anger, sadness, surprise, calmness, disgust and fear, and generate a time-synchronized facial expression label sequence.

[0091] The speech expression feature set is input into the speech emotion recognition network to identify the corresponding speech emotion category and generate a speech emotion label sequence that is time-synchronized with the facial expression label sequence. The matching accuracy is calculated by comparing the facial expression label sequence with the speech emotion label sequence. When the matching accuracy is lower than the threshold of 0.6, it is marked as an inconsistency between the facial expression and speech emotion of the character.

[0092] Based on the image semantic feature set, the character's lip contour and displacement trajectory data are extracted, and a lip movement time series is constructed. Based on the speech expression feature set, the vowel outbreak point, consonant transition segment and rhythm peak data are extracted to construct a speech pronunciation time series. The dynamic time warping algorithm is used to compare the lip movement time series and the speech pronunciation time series, and the synchronization offset rate is calculated. When the synchronization offset rate is greater than 12 frames, it is marked as an abnormal synchronization between the character's lip movement and speech pronunciation.

[0093] Step S200.2: Detecting the rationality of the physical movements of the person or object based on the image frame sequence, and detecting the rationality of the lens operation strategy based on the visual rhythm feature set.

[0094] The motion trajectory data of people and objects is extracted based on the image frame sequence difference algorithm. The optical flow estimation network and the object posture prediction network are used to calculate the continuity and physical rationality indicators of the motion trajectory, including speed change, direction change angle, and position offset. When the speed change exceeds 500 pixels / second, the direction change angle is greater than 90 degrees, or the position offset between two consecutive frames exceeds 100 pixels without a transition frame, it is marked as an unreasonable physical motion anomaly. Abnormal phenomena include floating, transients, and penetration.

[0095] The lens advancement rhythm features and lens switching density features in the visual rhythm feature set are obtained, and their conformity to the human eye's visual adaptation rules is analyzed. When the continuous frame segments with a lens switching interval of less than 200 milliseconds exceed 5 seconds, the lens advancement speed mutation rate exceeds 80%, or unnatural lens jitter lasts for more than 10 frames, it is marked as abnormal lens operation strategy. Abnormal lens operation strategy includes unnatural advancement, lens jumps without physical transitions, artificial handheld shaking, and non-smooth zooming.

[0096] Step S200.3: Detect the logical continuity of the background scene based on the scene label information set, detect the consistency of semantics and voice emotions based on the text semantic feature set and the voice expression feature set, and configure the micro video content consistency anomaly feature set.

[0097] A frame-sequence scene flow graph is constructed based on the scene label information set to analyze whether the continuous scene labels conform to the actual scene logic. When different logical scenes are switched continuously in a short period of time without any intermediate transition scenes, it is marked as a background scene logic break anomaly. The position continuity of the characters in different scenes is detected based on the scene relationship mapping graph. When it is detected that the position of the characters in different scenes changes instantly and there is a lack of reasonable movement process, it is marked as a character spatial position discontinuity anomaly.

[0098] The semantic type of the subtitle sentence is identified based on the text semantic feature set, and the corresponding voice tone type is identified based on the voice expression feature set. The consistency between the subtitle semantic type and the voice tone type is compared through the semantic tonality mapping table, and the consistency score is calculated. When the consistency score is lower than the threshold of 0.5, it is marked as an anomaly of inconsistency between semantics and voice emotion.

[0099] The anomalies such as inconsistency between facial expressions and voice emotions, asynchrony between lip movements and voice pronunciation, unreasonable physical movements, abnormal camera operation strategies, logical breaks in background scenes, discontinuous spatial positions of characters, and inconsistency between semantics and voice emotions are uniformly coded to generate a set of micro-video content consistency anomaly features, and the corresponding frame sequence positions and anomaly score values ​​are marked.

[0100] In some specific implementations, the step S300 is specifically: Step S300.1. Configure a compliance knowledge graph dedicated to AI marketing videos, match the semantic paths of the compliance knowledge graph based on the multimodal content fusion feature tensor, and obtain a path matching score.

[0101] A special compliance knowledge graph for AI marketing videos is constructed through system configuration. The special compliance knowledge graph for AI marketing videos is: a compliance rule mapping graph based on a graph neural network structure. The special compliance knowledge graph for AI marketing videos is composed of multiple compliance review semantic nodes and compliance rule paths between multiple semantic nodes.

[0102] The compliance review semantic nodes include: platform content review standard node, advertising law clause node, sensitive term definition node, competitor attack restriction clause node, false advertising judgment node and disclaimer specification node. Each compliance review semantic node is clearly associated with the corresponding text keyword, semantic sentence pattern, image scene attribute or voice expression feature. Multiple compliance review semantic nodes are connected based on the compliance constraint logical relationship to form a complete compliance rule path.

[0103] Based on the multimodal content fusion feature tensor structure, the semantic path matching engine is used to calculate the path matching score. The multimodal content fusion feature tensor structure is input into the semantic path matching engine. The semantic path matching engine is configured based on the graph attention network structure. The cross-modal semantic path matching task of text, image and speech features includes three matching dimensions: The first dimension: semantic matching is performed based on the subtitle sentence information extracted from the text semantic feature set and the keyword set of the platform content review standard node and sensitive term definition node, and the text compliance path matching results are obtained.

[0104] The second dimension: Based on the semantic mapping matching between the scene label information, brand logo pattern and product shape information extracted from the image semantic feature set and the competitor attack restriction clause node and false advertising judgment node, the image compliance path matching result is obtained.

[0105] The third dimension: Match the intonation change pattern extracted from the speech expression feature set with the speech emotion feature between the sensitive term definition nodes, and obtain the speech compliance path matching result.

[0106] For each compliance rule path, the semantic path matching engine uses the structural attention scoring function to calculate the path matching score value, specifically: ; Where: The path to be detected , is the number of nodes in the path, For the The fusion features extracted by the node, is the standard eigenvector of the node in the graph, is the graph node attention weight, Represents the similarity function between feature vectors, preferably cosine similarity or inverse normalized Euclidean distance function, Score value for path matching.

[0107] When the path matches the score value , and when the hit node satisfies the composite logical relationship, the path is marked as a high-risk semantic violation path.

[0108] Step S300.2: Identify the composite violation path based on the path matching score.

[0109] The system identifies compound violation paths through the path matching score of the semantic path matching engine and the multimodal feature combination relationship in the multimodal content fusion feature tensor. There are four conditions for compound violation path identification, namely: Condition 1: There is a medical environment label in the scene label information of the image semantic feature set, and there is a medical effect description in the subtitle text of the text semantic feature set, and no disclaimer subtitle is detected.

[0110] Condition 2: The speech expression feature set contains obvious exaggerated tone features, and marketing terms related to exaggerated tone are identified in the linkage relationship diagram between subtitle semantics and image body, and no disclaimer subtitles are detected.

[0111] Condition 3: There are competing product images in the image semantic feature set, and there are competitive comparisons or derogatory statements in the linkage relationship diagram between the subtitle semantics and the image body.

[0112] Condition 4: There are celebrity endorsement scene tags in the scene tag information set, and no corresponding non-professional disclaimer subtitles are detected.

[0113] When the composite violation path meets any of the four conditions and the path matching score exceeds 0.75, the system will mark the corresponding video segment as a composite violation.

[0114] Step S300.3: Generate a compliance assessment tag based on the violation path matching score value, and generate a structured violation path list and a preliminary compliance assessment result based on the compliance assessment tag.

[0115] The system classifies and structures the single violation paths and compound violation paths identified by the semantic path matching engine, and generates a compliance assessment label for each violation path. The system contains five compliance assessment labels, namely: Information on the compliance assessment label: 1. Violation types include: medical efficacy not clearly stated violation, visual and voice exaggeration violation, and competitor attack violation.

[0116] Information 2 of the compliance assessment label: the compliance review semantic node sequence corresponding to the violation path.

[0117] Information 3 of the compliance assessment label: Multimodal feature combination type triggered by the violation path, including: image violation features, text violation features, and voice violation features.

[0118] Information 4 of the compliance assessment label: the starting frame number and the ending frame number corresponding to the illegal video clip.

[0119] Information 5 of the compliance assessment label: Matching score value of the violation path.

[0120] Based on the matching score value and violation type severity in the compliance assessment tag, all violation paths and corresponding video clip information are organized into a structured violation path list. Based on the number of violation paths, violation type, and path matching score value, a preliminary compliance assessment result for the AI ​​marketing video is generated. The preliminary compliance assessment result includes: Level 1: Compliant, no illegal paths identified; Level 2: Minor violation, only one violation path with a matching score lower than 0.75 is identified; Level 3: Moderate violation, two or more violation paths with matching scores between 0.75 and 0.9 are identified; Level 4: Severe violation, identifying any violation path with a matching score value exceeding 0.9 or any compound violation path.

[0121] In some specific implementations, the step S400 is specifically: Step S400.1: Configure a subjective emotion tendency label set based on the speech expression feature set, and identify a marketing intention label sequence based on the text semantic feature set.

[0122] The speaking speed, tone, pitch, voice volatility and tone type extracted from the speech expression feature set are input into the multi-channel emotion recognition neural network. The multi-channel emotion recognition neural network adopts the acoustic feature and context attention fusion mechanism. The facial information of the characters in the image semantic feature set is input into the multi-label convolutional emotion recognition network. The multi-label convolutional emotion recognition network performs facial expression classification on each frame. Through the emotion consistency fusion strategy, the output results of the multi-channel emotion recognition neural network are fused with the output results of the multi-label convolutional emotion recognition network to generate a time-synchronized set of subjective emotion tendency labels.

[0123] The set of subjective emotion tendency labels includes: joy, elation, calmness, neutrality, depression, sadness, anger and anxiety. Each frame of the video corresponds to an emotion tendency label and confidence score.

[0124] The frame-synchronized subtitle sentences in the text semantic feature set are input into the marketing intention recognition network. The marketing intention recognition network is a semantic intent classification model based on the bidirectional contextual attention mechanism. The marketing intention recognition network outputs the marketing intention label corresponding to each frame subtitle sentence according to the sentence structure, keyword weight and semantic directionality. The marketing intention labels include: product recommendation, benefit-driven, emotional infection, professional endorsement and usage scenario guidance. The marketing intention labels correspond to the video frame sequence timeline and form a marketing semantic-oriented structure of the video.

[0125] Step S400.2: Configure a conflict scoring function between the emotion tendency tag set and the marketing intention tag sequence, and obtain an emotion semantic conflict score value.

[0126] Based on the subjective emotion tendency label set and marketing intention label sequence, four typical conflict judgment combinations are set, namely: Combination 1: When the subjective emotional tendency label is sadness or depression, and the marketing intention label is profit-driven or emotional contagion, it is defined as a high-level conflict.

[0127] Combination 2: When the subjective emotional tendency label is anger or anxiety, and the marketing intention label is usage scenario guidance or professional endorsement, it is defined as a high-level conflict.

[0128] Combination 3: When the subjective emotional tendency label is calm or neutral, and the marketing intention label is emotional contagion, it is defined as medium-level conflict.

[0129] Combination 4: When the subjective emotion tendency label matches the marketing intention label, it is defined as non-conflict.

[0130] Define the sentiment semantic conflict scoring function as follows: ; Where: For the The emotional semantic conflict score of the frame, is the confidence change rate of emotional tendency, , is the semantic change rate of the marketing intention label, and its value is , Subjective emotional tendency label Tags with marketing intent The conflict level scores are 0 no conflict, 0.5 medium conflict, and 1 high conflict. is the weighting coefficient, and the preferred value is , the sum of the three is 1.

[0131] Step S400.3: Identify misleading expression segments based on the emotion-semantic conflict score, and generate an emotion-semantic conflict risk label and a structured conflict risk data list.

[0132] The emotional semantic conflict score value calculation is performed on the entire video frame. When the emotional semantic conflict score value of any frame , it is marked as a high-risk frame for potential misleading expression, and a window sliding detection method is used in the frame sequence timeline. If the number of high conflict score frames exceeds 45 frames continuously, the corresponding frame segment is identified as a potential misleading expression segment, and the start and end time information of the current frame segment is output.

[0133] For all segments identified as potentially misleading expressions, an emotional semantic conflict risk label is generated. The emotional semantic conflict risk label includes: the combination type of conflict between emotional tendency and marketing intention, the start and end frame numbers of the video segment, the maximum conflict score, the average semantic jump rate, the maximum emotional confidence fluctuation value, the conflict level, and whether it covers the main sentence expression.

[0134] In some specific implementations, the step S500 is specifically: Step S500.1: Aggregate multiple types of violation identification results to form a violation risk feature matrix, and track the violation causal path and locate the violation trigger source.

[0135] The micro-video content consistency anomaly feature set, structured violation path list and emotional semantic conflict risk label are time-synchronized and feature-merged to form a frame-level violation risk feature matrix. Each row of the violation risk feature matrix contains: video frame number, violation risk type, violation risk label, violation score value, risk level, violation reason text, violation trigger mode type and violation feature point frame index.

[0136] Step S500.2: Mark high-risk frame segments according to the calculation results.

[0137] For each video frame, the total violation risk score is calculated by weighting the micro-video content consistency anomaly feature set, compliance knowledge graph, and emotional semantic conflict score in the violation risk feature matrix. Specifically, the total violation risk score is: ; Where: It is The total violation risk score value of the frame, is a collection of abnormal features of microscopic video content consistency, It is a compliance knowledge graph. is the emotional semantic conflict score, 、 、 is the weighting coefficient, and its value is 、 、 .

[0138] when When , it is marked as a high-risk frame. When , it is marked as a medium-risk frame, and the rest are low-risk frames. Continuous high-risk frame segments are merged into high-risk video segments, and the start and end frame sequence indexes are recorded.

[0139] Step S500.3: Perform causal path tracing of violation behaviors and identification of trigger sources, generate structured violation modification suggestions and mark applicable areas, and output an explainable violation risk report and a structured modification suggestion list.

[0140] For all high-risk video clips, the violation causal path tracing algorithm is called to use the violation type, violation label, violation trigger mode type, violation description text and violation feature point frame index information marked in the violation risk feature matrix, combined with the compliance knowledge graph to perform path backtracking, extract the violation trigger chain and the starting causal point, and output the violation event causal path structure. The violation event causal path structure includes: violation type, first trigger frame number, violation source mode, associated violation content description text, violation feature point frame index, hit compliance knowledge graph clause node sequence and the start and end frame sequence range of the high-risk frame segment involved.

[0141] Based on the violation type, violation source mode, associated violation content description text and hit compliance knowledge graph clause node sequence extracted from the causal path structure of the violation event, a structured violation modification suggestion list is constructed. The suggestions include: recommended operation type, recommended content description, recommended applicable frame segment start and end frame sequence numbers, involved violation reason text, based on the compliance knowledge graph clause node number and rule content and expected violation risk score reduction range.

[0142] The structured violation modification suggestion list is output in JSON structure and table structure respectively.

[0143] Based on the violation risk feature matrix, the total violation risk score, and the structured violation modification suggestion list, an explainable violation risk report in a standard format is generated, including: a full-frame violation risk heat map of the video, showing the trend of frame-level violation score changes, a statistical chart of violation events classified by violation type and a pie chart of violation ratios, an index of the start and end frames of all high-risk clips, the violation types and violation labels involved, a visualization map of the causal path of violation events, a structured violation modification suggestion list, and the total violation risk score and platform release suggestions.

[0144] The report is output in PDF format and WebJSON structure respectively.

[0145] In some specific implementations, the step S600 is specifically: Step S600.1: Configure the multi-dimensional scoring dimensions of the AI ​​marketing video quality grading evaluation model and configure the AI ​​marketing video quality comprehensive scoring function.

[0146] Configure an AI marketing video quality grading and evaluation model. The AI ​​marketing video quality grading and evaluation model performs multi-dimensional quality scoring based on the fusion of multi-source feature information input. The input multi-source feature information includes: multimodal content fusion feature tensor, micro-video content consistency anomaly feature set, emotional semantic conflict score, path matching score and platform communication feedback indicator set. The platform communication feedback indicator set includes four core feedback parameters: video completion rate, number of likes, comment interaction frequency and content forwarding number.

[0147] Five core scoring dimensions are constructed based on the input multi-source feature information, namely: video naturalness scoring item, visual coherence scoring item, emotional marketing consistency scoring item, violation risk scoring item and platform communication feedback scoring item.

[0148] Each scoring item adopts a normalized scoring mechanism with a score range of [0,1], and a comprehensive scoring function is introduced to calculate the comprehensive score value.

[0149] Set the video quality comprehensive scoring function to: ; Where: is the comprehensive quality score of AI marketing videos. is the video naturalness rating value, is the visual coherence score, Emotional marketing consistency score, is the violation risk score value, Provide feedback ratings for platform dissemination, is the weighted coefficient corresponding to each scoring item, configured as and 0.2, satisfying that the sum of weighted coefficients is 1.

[0150] Step S600.2: Generate a quality grade label based on the comprehensive scoring function result and output a quality assessment report.

[0151] The quality level of AI marketing videos is marked based on the comprehensive score. The classification criteria include: When , it is marked as high quality, when When , it is marked as medium quality, when , marked as low quality.

[0152] Generates a structured quality assessment report, which includes: video quality grade labels, detailed rating values ​​for each rating item, a heat map of video frame-level violation risks, and a summary of marketing sentiment and communication effect analysis.

[0153] The structured quality assessment report will be output in WebJSON structure and PDF format respectively.

[0154] For high-quality videos, a green release mark is automatically assigned, the system review depth is set to the minimum, and only risk spot checks are performed. For medium-quality videos, the system triggers a local enhanced review process, focusing on detecting risky frames and semantically deviated segments. For low-quality videos, the system outputs detailed quality optimization suggestions to the content publisher and temporarily suspends the push process until corrections are made and the score is re-scored.

[0155] In the above content, in actual application, first, after receiving the AI ​​marketing video to be reviewed, the multimodal video content parsing model deployed on the review server is called to parse the video frame by frame, extract the image frame sequence, voice track, subtitle text and background scene image, and for the image frame sequence, extract static visual information, character action features and lens motion parameters; for the voice track, extract the intonation, speaking speed and pronunciation consistency features; for the subtitle text, extract the text semantic embedding vector and sentiment tendency label; for the background image, extract the scene category and composition elements. After the extraction is completed, the above features are encoded into a fused multimodal content expression tensor through a unified feature mapping channel, which serves as the input basis for subsequent content consistency detection and violation analysis.

[0156] After completing the multimodal feature extraction, a six-dimensional perceptual consistency detection mechanism is constructed based on the principle of natural continuity of video and human viewing habits. The six-dimensional detection content includes: matching detection of character expressions and voice intonation, detection of lip shape and pronunciation synchronization, detection of consistency between actions and physical laws, detection of whether there is unnatural movement in the lens advancement strategy, analysis of the logical coherence of background scenes, and judgment of the expression consistency between emotional expression and marketing semantics. The above detection process generates a consistency score vector through rule encoding and anomaly scoring algorithm, with a unified standard of floating-point scores between 0 and 1. Dimension scores below 0.3 are judged as significant anomalies, providing basic data support for subsequent violation path analysis and risk scoring.

[0157] Then, based on the preset AI marketing video-specific compliance knowledge graph, the knowledge graph nodes include platform rules and regulations, corresponding provisions of the Advertising Law, common sensitive expression vocabulary, industry constraint expressions, and false propaganda criteria. The semantic features in the fused content expression tensor are vector matched to identify potential violation paths. The matching method adopts multi-hop path search and semantic attention weighted mechanism, which can identify complex violation structures synthesized across modalities, such as emotionally inflammatory tone combined with excessive efficacy descriptions, etc., and finally output the initial compliance judgment label and generate a graph path matching record for causal path reconstruction and risk positioning of illegal content.

[0158] Next, the tone, speaking speed and facial expression features extracted from the video are input into the trained emotion recognition network to generate standard emotion category labels. Combined with the text semantic recognition module, the type of marketing purpose is determined, including emotional appeal, interest stimulation, rational explanation, etc. The output results of the two are compared, and an emotional semantic consistency score is generated based on the preset conflict judgment matrix. If there are obvious conflicting expressions such as "sad expression combined with passionate promotional slogans" and the score is lower than 0.4, it will be automatically marked as high-risk conflict content, providing an important reference path for subsequent judgment of illegal content.

[0159] Secondly, the consistency detection results, compliance assessment labels, and emotional semantic conflict score results are aggregated to construct a violation risk score matrix at the frame level. The violation causal path tracing algorithm is used to locate the specific violation clips, and the four elements of violation trigger source content, violation reason, graph path node, and violation type are output. According to different types of violation content, the preset structured suggestion template library is called to output operational guidelines including word replacement suggestions, tone adjustment suggestions, camera switching plans, and subtitle modification plans. Finally, an audit report with interpretability and executable nature is generated. The report marks the timestamp of the violation clip and is accompanied by the modification threshold conditions corresponding to each type of suggestion. For example, when the violation risk score is greater than 0.7, the word content is forced to be modified.

[0160] Finally, the fused multimodal content expression tensor, consistency score vector, violation risk score matrix and graph matching results are input into the quality assessment model to generate a comprehensive score for the marketing video. The assessment model outputs the video quality level, which is divided into three levels: low quality score <0.4, medium quality 0.4≤ score <0.75 and high quality score ≥0.75. Different review response strategies are selected according to the quality level. For example, low-quality videos trigger the re-review process, high-quality videos are automatically pushed to the pre-release platform, and the optimization process is triggered according to the recommended content. The entire processing flow forms a closed-loop execution logic from data analysis, compliance review, risk positioning to optimization feedback.

[0161] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. The AI-based marketing video review method is characterized by: The steps include: S100, obtaining a set of original multimodal data structures, extracting an image semantic feature set, a visual rhythm feature set, a speech expression feature set, an audio-visual synchronization association vector structure, a text semantic feature set, a subtitle semantics and image subject linkage relationship diagram, a scene label information set, and a scene relationship mapping diagram, and configuring a multimodal content fusion feature tensor structure; S200, based on the feature set, detecting the matching degree between the character's expression and voice emotion, the lip shape and voice synchronization, the rationality of the action, the rationality of the camera movement, the logical continuity of the scene, and the semantic emotion consistency, and configuring the micro-video content consistency abnormality feature set; S300, configures a compliance knowledge graph dedicated to AI marketing videos, uses multimodal content fusion feature tensors for semantic path matching, identifies complex violation paths, and generates a structured violation path list and preliminary compliance assessment results; S400: Configure a subjective emotion tendency label set and a marketing intention label sequence, calculate an emotion semantic conflict score, identify misleading expression segments, and generate an emotion semantic conflict risk label and a structured conflict risk data list; S500: Aggregate multiple violation identification results to form a violation risk feature matrix, track the triggering sources of violations, and generate an explainable violation risk report and a structured modification suggestion list; S600: Configure multi-dimensional scoring dimensions and comprehensive scoring functions, and output quality grade labels and quality assessment reports.

2. The AI-based marketing video review method according to claim 1, characterized in that: The S100 is specifically: S100.

1. Obtain a set of original structures of multimodal data and extract image semantic features; The system accepts an AI marketing video for review, wherein the AI ​​marketing video includes: image frame sequences, voice tracks, subtitle text, and background scene data channels; Input the AI ​​marketing video into the video structure decoding module, perform decoding processing, extract the time-synchronized image frame sequence, audio track and subtitle layer, and annotate the relationship between frames to form a multimodal data original structure set; The image frame sequence is input into the image semantic feature extraction network. The image semantic feature extraction network uses a trained multi-scale convolutional neural network structure to analyze the facial information of the characters, the shape of the products, the brand logo pattern, the primary and secondary layout of the image, and the dynamic relationship of the action. It performs semantic encoding processing on each frame of the image to obtain a set of image semantic features. Furthermore, the motion analysis operation between image frame sequences is performed, and the lens motion features, lens advancement rhythm features and lens switching density features between image frame sequences are extracted using the image frame sequence difference analysis algorithm, the bidirectional optical flow estimation algorithm and the key frame judgment algorithm, thereby forming a visual rhythm feature set of the image frame sequence. S100.

2. Extracting speech expression features based on the original structure set of multimodal data; The audio track is fed into a speech expression feature extraction network. This network uses an emotion modeling network based on a deep acoustic modeling structure to extract speech rate, pitch, tone, voice volatility, and tone type from the human voice, and forms a speech expression feature set. Synchronously aligning the speech expression feature set with the visual rhythm feature set of the image frame sequence in the image frame sequence to form an audio-visual synchronization association vector structure; S100.

3. Extracting text semantic features based on the original structure set of multimodal data; Perform text detection and recognition operations on each frame of the subtitle layer, call the video text recognition module to extract the subtitle content and advertising description, and output a time-sequenced subtitle text sequence corresponding to the frame sequence. The subtitle text sequence is input into the text semantic feature extraction network. The text semantic feature extraction network adopts a natural language understanding model based on contextual semantic modeling to identify promotional verbs, functional descriptive words, suggestive evaluation words and risk descriptions, and performs semantic embedding coding on each identified subtitle sentence to extract a set of text semantic features. Furthermore, based on subtitle recognition, the spatial relationship between the subtitle position and the main object of the picture is modeled to construct a linkage relationship diagram between the subtitle semantics and the main image. S100.

4. Extract scene label information based on the original structure set of multimodal data, and configure a multimodal content fusion feature tensor; The image frame sequence is input into a scene label extraction network. The scene label extraction network is a trained image classification neural network that performs frame-by-frame scene recognition on the image frame sequence. The recognition includes: medical environment, school environment, home space, public passage or commercial store, and generates a corresponding scene label information set; Further identify whether there are background brand logo patterns, competing product patterns, and the positional relationship between people and main objects in the scene in the scene image, and construct a scene relationship map; The image semantic feature set, visual rhythm feature set, speech expression feature set, audio-visual synchronization association vector structure, text semantic feature set, subtitle semantics and image subject linkage relationship diagram, scene label information set and scene relationship mapping diagram are uniformly encoded and mapped into a unified multimodal feature space using a fusion embedding mechanism, and a complete multimodal content fusion feature tensor structure is constructed.

3. The AI-based marketing video review method according to claim 1, characterized in that: The S200 is specifically: S200.

1. Detect the degree of matching between facial expressions and speech emotions, and the synchronization between lip movements and speech pronunciation, based on the image semantic feature set and the speech expression feature set; The facial information of the characters in the image semantic feature set is input into a multi-label convolutional emotion recognition network to identify the facial expression categories of the characters, including joy, anger, sadness, surprise, calmness, disgust and fear, and generate a time-synchronized sequence of facial expression labels; The speech expression feature set is input into the speech emotion recognition network to identify the corresponding speech emotion category and generate a speech emotion label sequence that is time-synchronized with the facial expression label sequence. The facial expression label sequence and the speech emotion label sequence are compared to calculate the matching accuracy. When the matching accuracy falls below a threshold of 0.6, it is marked as an inconsistency between the facial expression and speech emotion of the person. Based on the image semantic feature set, the character's lip shape contour and displacement trajectory data are extracted, and a lip movement time series is constructed. Based on the speech expression feature set, the vowel burst point, consonant transition segment and rhythm peak data are extracted to construct a speech pronunciation time series. The dynamic time warping algorithm is used to compare the lip movement time series with the speech pronunciation time series, and the synchronization offset rate is calculated. When the synchronization offset rate is greater than 12 frames, it is marked as an abnormal synchronization between the character's lip movement and speech pronunciation. S200.

2. Detecting the rationality of the physical movements of people or objects based on the image frame sequence, and detecting the rationality of the camera movement strategy based on the visual rhythm feature set; Using an image frame sequence difference algorithm to extract motion trajectory data for people and objects, the optical flow estimation network and the object pose prediction network are used to calculate trajectory continuity and physical plausibility indicators, including speed change, direction change angle, and position offset. When the speed change exceeds 500 pixels / second, the direction change angle is greater than 90 degrees, or the position offset between two consecutive frames exceeds 100 pixels with no transition frames, it is marked as an unreasonable physical motion anomaly. Abnormal phenomena include floating, transients, and penetration. Obtain the shot advancement rhythm features and shot switching density features from the visual rhythm feature set and analyze whether they conform to the laws of human visual adaptation. If the continuous frame segments with a shot switching interval of less than 200 milliseconds exceed 5 seconds, the shot advancement speed mutation rate exceeds 80%, or unnatural camera shake persists for more than 10 frames, it is marked as abnormal camera operation strategy. Abnormal camera operation strategies include unnatural advancement, shot jumps without physical transitions, artificial handheld shaking, and non-smooth zooming. S200.

3. Detecting the logical continuity of the background scene based on the scene label information set, detecting the consistency of semantics and speech emotions based on the text semantic feature set and the speech expression feature set, and configuring a micro-video content consistency anomaly feature set; Based on the scene label information set, a frame sequence scene flow graph is constructed to analyze whether the continuous scene labels conform to the actual scene logic. When different logical scenes are switched continuously in a short period of time without intermediate transition scenes, it is marked as a background scene logic break anomaly. The position continuity of the characters in different scenes is detected based on the scene relationship map. When the position of the characters in different scenes is detected to change instantly and there is no reasonable movement process, it is marked as a character spatial position discontinuity anomaly. The semantic type of the subtitle sentence is identified based on the text semantic feature set, and the corresponding voice tone type is identified based on the voice expression feature set. The consistency between the subtitle semantic type and the voice tone type is compared using the semantic tonality mapping table, and a consistency score is calculated. When the consistency score falls below the threshold of 0.5, it is marked as an anomaly of inconsistency between semantics and voice emotion; The anomalies such as inconsistency between facial expressions and voice emotions, asynchrony between lip movements and voice pronunciation, unreasonable physical movements, abnormal camera operation strategies, logical breaks in background scenes, discontinuous spatial positions of characters, and inconsistency between semantics and voice emotions are uniformly coded to generate a set of micro-video content consistency anomaly features, and the corresponding frame sequence positions and anomaly score values ​​are marked.

4. The AI-based marketing video review method according to claim 1, characterized in that: The S300 is specifically as follows: S300.

1. Configure a compliance knowledge graph dedicated to AI marketing videos, match semantic paths in the compliance knowledge graph based on multimodal content fusion feature tensors, and derive a path matching score. Constructing a compliance knowledge graph dedicated to AI marketing videos through system configuration. The compliance knowledge graph dedicated to AI marketing videos is: a compliance rule mapping graph based on a graph neural network structure. The compliance knowledge graph dedicated to AI marketing videos is composed of multiple compliance review semantic nodes and compliance rule paths between multiple semantic nodes; The compliance review semantic nodes include: platform content review standard node, advertising law clause node, sensitive term definition node, competitor attack restriction clause node, false advertising determination node, and disclaimer specification node. Each compliance review semantic node is clearly associated with a corresponding text keyword, semantic sentence pattern, image scene attribute, or voice expression feature. Multiple compliance review semantic nodes are connected based on compliance constraint logical relationships to form a complete compliance rule path. Based on the multimodal content fusion feature tensor structure, the semantic path matching engine is used to calculate the path matching score. The multimodal content fusion feature tensor structure is input into the semantic path matching engine. The semantic path matching engine is configured based on the graph attention network structure. The cross-modal semantic path matching task of text, image and speech features includes three matching dimensions: The first dimension: semantic matching is performed based on the subtitle sentence information extracted from the text semantic feature set and the keyword set of the platform content review standard node and sensitive term definition node, and the text compliance path matching result is obtained; The second dimension: Based on the semantic mapping between the scene label information, brand logo pattern and product form information extracted from the image semantic feature set and the competitor attack restriction clause node and false advertising determination node, the image compliance path matching result is obtained; The third dimension: Matching the intonation variation patterns extracted from the speech expression feature set with the speech emotion features between the sensitive term definition nodes, and obtaining the speech compliance path matching results; For each compliance rule path, the semantic path matching engine uses the structural attention scoring function to calculate the path matching score value, specifically: ； Where: The path to be detected , is the number of nodes in the path, For the The fusion features extracted by the node, is the standard eigenvector of the node in the graph, is the graph node attention weight, Represents the similarity function between feature vectors, preferably cosine similarity or inverse normalized Euclidean distance function, Score value for path matching; When the path matches the score value , and when the hit node satisfies the composite logical relationship, the path is marked as a high-risk semantic violation path; S300.

2. Identify the path of compound violation behavior based on the path matching score; The system identifies compound violation paths through the path matching score of the semantic path matching engine and the multimodal feature combination relationship in the multimodal content fusion feature tensor. There are four conditions for compound violation path identification, namely: Condition 1: The scene label information of the image semantic feature set contains a medical environment label, and the subtitle text of the text semantic feature set contains a description of the medical effect, and no disclaimer subtitle is detected; Condition 2: The speech expression feature set contains obvious exaggerated tone features, and the subtitle semantics and image body linkage relationship diagram identifies marketing terms related to exaggerated tone, and no disclaimer subtitles are detected; Condition 3: The image semantic feature set contains images of competing products, and the caption semantics and image subject linkage diagram contains competitive comparisons or derogatory statements; Condition 4: There are celebrity endorsement scene tags in the scene tag information set, and no corresponding non-professional disclaimer subtitles are detected; When the composite violation path meets any of the four conditions and the path matching score exceeds 0.75, the system will mark the corresponding video segment as a composite violation; S300.

3. Generate a compliance assessment tag based on the violation path matching score value, and generate a structured violation path list and preliminary compliance assessment results based on the compliance assessment tag; The system classifies and structures the single violation paths and compound violation paths identified by the semantic path matching engine, and generates a compliance assessment label for each violation path. The system contains five compliance assessment labels, namely: Information on the Compliance Assessment Label:

1. Violation types include: failure to clearly state medical efficacy, visual and audio exaggeration, and competitor attack. Information 2 of the compliance assessment label: the compliance review semantic node sequence corresponding to the violation path; Compliance assessment label information 3: The type of multimodal feature combination triggered by the violation path, including image violation features, text violation features, and voice violation features; Information 4 of the compliance assessment label: the starting frame number and ending frame number of the offending video clip; Information 5 of the compliance assessment tag: the matching score value of the violation path; Based on the matching score value and violation type severity in the compliance assessment tag, all violation paths and corresponding video clip information are organized into a structured violation path list. Based on the number of violation paths, violation type, and path matching score value, a preliminary compliance assessment result for the AI ​​marketing video is generated. The preliminary compliance assessment result includes: Level 1: Compliant, no illegal paths identified; Level 2: Minor violation, only one violation path with a matching score lower than 0.75 is identified; Level 3: Moderate violation, two or more violation paths with matching scores between 0.75 and 0.9 are identified; Level 4: Severe violation, identifying any violation path with a matching score value exceeding 0.9 or any compound violation path.

5. The AI-based marketing video review method according to claim 1, characterized in that: The S400 is specifically: S400.

1. Configure a subjective emotion tendency label set based on the speech expression feature set, and identify a marketing intention label sequence based on the text semantic feature set; The speech rate, tone, pitch, voice volatility, and tone type extracted from the speech expression feature set are input into a multi-channel emotion recognition neural network. The multi-channel emotion recognition neural network uses an acoustic feature and contextual attention fusion mechanism. The facial information of the characters in the image semantic feature set is input into a multi-label convolutional emotion recognition network. The multi-label convolutional emotion recognition network performs facial expression classification on each frame. The output results of the multi-channel emotion recognition neural network are fused with the output results of the multi-label convolutional emotion recognition network through an emotion consistency fusion strategy to generate a time-synchronized set of subjective emotion tendency labels. The subjective emotion tendency label set includes: joy, elation, calmness, neutrality, depression, sadness, anger, and anxiety. Each frame of the video corresponds to an emotion tendency label and confidence score. The frame-synchronized subtitle sentences in the text semantic feature set are input into the marketing intent recognition network. The marketing intent recognition network is a semantic intent classification model based on a bidirectional contextual attention mechanism. The marketing intent recognition network outputs the marketing intent label corresponding to each frame subtitle sentence based on sentence structure, keyword weight, and semantic directionality. Marketing intent labels include: product recommendation, profit-driven, emotional influence, professional endorsement, and usage scenario guidance. The marketing intent labels correspond to the video frame timeline and form a marketing semantic guidance structure for the video. S400.

2. Configure a conflict scoring function between the emotion tendency tag set and the marketing intention tag sequence, and obtain an emotion semantic conflict score value; Based on the subjective emotion tendency label set and marketing intention label sequence, four typical conflict judgment combinations are set, namely: Combination 1: When the subjective emotional tendency label is sadness or depression, and the marketing intention label is profit-driven or emotional contagion, it is defined as high-level conflict; Combination 2: When the subjective emotion tendency label is anger or anxiety, and the marketing intention label is usage scenario guidance or professional endorsement, it is defined as high-level conflict; Combination 3: When the subjective emotional tendency label is calm or neutral, and the marketing intention label is emotional contagion, it is defined as medium-level conflict; Combination 4: When the subjective emotion tendency label matches the marketing intention label, it is defined as no conflict; Define the sentiment semantic conflict scoring function as follows: ； Where: For the The emotional semantic conflict score of the frame, is the confidence change rate of emotional tendency, , is the semantic change rate of the marketing intention label, and its value is , Subjective emotional tendency label Tags with marketing intent The conflict level scores are 0 no conflict, 0.5 medium conflict, and 1 high conflict. is the weighting coefficient, and the preferred value is , the sum of the three is 1; S400.

3. Identify misleading expression segments based on the sentiment-semantic conflict score, and generate sentiment-semantic conflict risk labels and a structured conflict risk data list; The emotional semantic conflict score value calculation is performed on the entire video frame. When the emotional semantic conflict score value of any frame , it is marked as a high-risk frame for potential misleading expression, and a window sliding detection method is used in the frame sequence timeline. If the number of high conflict score frames exceeds 45 frames in a row, the corresponding frame segment is identified as a potential misleading expression segment, and the start and end time information of the current frame segment is output; For all segments identified as potentially misleading expressions, an emotional semantic conflict risk label is generated. The emotional semantic conflict risk label includes: the combination type of conflict between emotional tendency and marketing intention, the start and end frame numbers of the video segment, the maximum conflict score, the average semantic jump rate, the maximum emotional confidence fluctuation value, the conflict level, and whether it covers the main sentence expression.

6. The AI-based marketing video review method according to claim 1, characterized in that: The S500 is specifically: S500.

1. Aggregate multiple violation identification results to form a violation risk feature matrix, and then trace the violation causal path and locate the triggering source of the violation; The micro-video content consistency anomaly feature set, structured violation path list, and emotional semantic conflict risk label are time-series synchronized and feature-merged to form a frame-level violation risk feature matrix. Each row of the violation risk feature matrix contains: video frame number, violation risk type, violation risk label, violation score value, risk level, violation reason text, violation trigger mode type, and violation feature point frame index; S500.

2. Mark high-risk frame segments based on the calculation results; For each video frame, the total violation risk score is calculated by weighting the micro-video content consistency anomaly feature set, compliance knowledge graph, and emotional semantic conflict score in the violation risk feature matrix. Specifically, the total violation risk score is: ； Where: It is The total violation risk score value of the frame, is a collection of abnormal features of microscopic video content consistency, It is a compliance knowledge graph. is the emotional semantic conflict score, 、 、 is the weighting coefficient, and its value is 、 、 ; when When , it is marked as a high-risk frame. When , it is marked as a medium-risk frame, and the rest are low-risk frames. Continuous high-risk frame segments are merged into high-risk video segments, and the start and end frame sequence indexes are recorded; S500.

3. Perform causal path tracing and trigger source identification for violations, generate structured violation modification suggestions and mark applicable areas, and output an explainable violation risk report and a list of structured modification suggestions; For all high-risk video clips, the violation causal path tracing algorithm is called. The violation type, violation label, violation trigger mode type, violation description text and violation feature point frame index information marked in the violation risk feature matrix are used in combination with the compliance knowledge graph to perform path backtracking, extract the violation trigger chain and the starting causal point, and output the violation event causal path structure. The violation event causal path structure includes: violation type, first trigger frame number, violation source mode, associated violation content description text, violation feature point frame index, hit compliance knowledge graph clause node sequence and the start and end frame sequence range of the high-risk frame segment involved; Based on the violation type, violation source modality, associated violation content description text, and the hit compliance knowledge graph clause node sequence extracted from the causal path structure of the violation event, a structured violation modification suggestion list is constructed. The suggestion includes: the recommended operation type, the recommended content description, the recommended applicable frame segment start and end frame sequence, the violation reason text involved, the compliance knowledge graph clause node number and rule content based on it, and the expected violation risk score reduction range; The structured violation modification suggestion list is output in JSON structure and table structure respectively; Based on the violation risk feature matrix, total violation risk score, and structured violation modification suggestion list, a standard format, explainable violation risk report is generated. This report includes: a full-frame violation risk heat map showing the trend of frame-level violation score changes, a statistical chart of violation events classified by violation type and a pie chart of violation ratios, an index of the start and end frames of all high-risk clips, the violation types and violation labels involved, a visualization of the causal path of violation events, a structured violation modification suggestion list, a total violation risk score, and platform release suggestions; The report is output in PDF format and WebJSON structure respectively.

7. The AI-based marketing video review method according to claim 1, characterized in that: The S600 is specifically: S600.

1. Configure the multi-dimensional scoring dimensions of the AI ​​marketing video quality grading assessment model and the comprehensive scoring function for AI marketing video quality. Configure an AI marketing video quality grading and assessment model. This model performs multi-dimensional quality scoring based on the fusion of multi-source feature information. The input multi-source feature information includes: a multimodal content fusion feature tensor, a micro-video content consistency anomaly feature set, an emotional semantic conflict score, a path matching score, and a platform communication feedback indicator set. The platform communication feedback indicator set includes four core feedback parameters: video completion rate, number of likes, comment interaction frequency, and content forwarding count. Five core scoring dimensions are constructed based on the input multi-source feature information, namely: video naturalness scoring, visual coherence scoring, emotional marketing consistency scoring, violation risk scoring, and platform communication feedback scoring. Each scoring item adopts a normalized scoring mechanism with a score range of [0,1], and introduces a comprehensive scoring function to calculate the comprehensive score value; S600.

2. Generate a quality grade label based on the comprehensive scoring function results and output a quality assessment report; The quality level of AI marketing videos is marked based on the comprehensive score. The classification criteria include: When , it is marked as high quality, when When , it is marked as medium quality, when When , it is marked as low quality; Generate a structured quality assessment report, which includes: video quality grade labels, detailed rating values ​​for each rating item, a heat map of video frame-level violation risks, and a summary of marketing sentiment and communication effectiveness analysis; The structured quality assessment report will be output in WebJSON structure and PDF format respectively; For high-quality videos, a green release mark is automatically assigned, the system review depth is set to the minimum, and only risk spot checks are performed. For medium-quality videos, the system triggers a local enhanced review process, focusing on detecting risky frames and semantically deviated segments. For low-quality videos, the system outputs detailed quality optimization suggestions to the content publisher and temporarily suspends the push process until corrections are made and the score is re-scored.

Images