AI-based marketing video review method

Abstract

The AI-based marketing video auditing method provided in the application relates to the technical field of AI marketing video auditing and comprises the following steps: acquiring a multi-modal data original structure set, extracting image semantic features, speech expression features, text semantic features and scene label information, obtaining an image semantic feature set, a visual rhythm feature set, a speech expression feature set, an audio-visual synchronization correlation vector structure, a text semantic feature set and a subtitle semantic and image main body linkage relationship graph. By constructing the image semantic feature set, the speech expression feature set, the text semantic feature set and the visual rhythm feature set and fusing them into a multi-modal content fusion feature tensor, unified modeling of AI marketing videos in the visual, auditory and semantic layers can be realized, and effective micro-consistency detection, compliance knowledge graph and emotional semantic conflict identification can be provided for subsequent video content full-link risk perception and accurate auditing.

Description

TECHNICAL FIELD

[0001] The application relates to the field of AI marketing video auditing technology, in particular to an AI-based marketing video auditing method. BACKGROUND

[0002] With the development of artificial intelligence and short video platforms, AI-generated marketing videos have gradually become an important means of digital marketing. Compared with traditional advertising materials, AI marketing videos have advantages such as high generation efficiency, diversified content, and accurate semantic expression, and are widely used in e-commerce promotion, product recommendation, and brand promotion scenarios. Current mainstream auditing strategies mainly rely on manual annotation, rule templates, or machine learning models based on single-modal content for judgment, which cannot comprehensively handle complex compliance, emotional orientation, and subjective expression conflicts in multi-modal content. Therefore, there is an urgent need to build an intelligent auditing method based on multi-modal feature understanding and behavior semantic recognition to improve the ability of AI marketing videos in automatic, structured, and interpretable auditing.

[0003] However, the existing AI video auditing method has the following key defects: first, it lacks a deep semantic fusion mechanism for multi-modal content such as images, speech, and text, which makes the auditing system unable to accurately identify the linkage information between different modalities, leading to misjudgment or missed judgment; second, it lacks systematic analysis of the consistency between emotional expression and marketing intent, especially in videos with subjective emotional color, which easily ignores potential misleading expressions, causing content-oriented risks. These defects reduce the accuracy and interpretability of the auditing results in the actual auditing process, and also limit the platform's compliance supervision ability for marketing content. SUMMARY

[0004] The present application aims to overcome the shortcomings of the prior art and provide an AI-based marketing video auditing method. By constructing image semantic feature sets, speech expression feature sets, text semantic feature sets, visual rhythm feature sets, and multi-modal content fusion feature tensors, the method realizes deep fusion analysis of multi-modal content. Further, by combining facial expression and voice emotion consistency detection, marketing intent recognition, and emotional semantic conflict scoring models, the method realizes accurate identification of potential misleading content in videos. Through compliance knowledge graph path matching scoring mechanisms and structured violation causal path generation methods, the method improves the logical reasoning ability and compliance recognition coverage of the auditing system. Finally, the system constructs an AI marketing video quality grading evaluation model based on the above content, realizes the integration and closed-loop processing of auditing and quality evaluation, and significantly improves the auditing effect of AI marketing videos in terms of compliance, accuracy, and interpretability.

[0005] To achieve the above-mentioned purposes, the present application provides the following technical solutions:

[0006] The AI-based marketing video auditing method comprises the following steps:

[0007] In step S100, a multi-modal data original structure set is obtained, and an image semantic feature set, a visual rhythm feature set, a speech expression feature set, an audio-visual synchronization correlation vector structure, a text semantic feature set, a subtitle semantic and image main body linkage graph, a scene label information set and a scene relationship mapping graph are extracted, and a multi-modal content fusion feature tensor structure is configured.

[0008] In step S200, the matching degree of facial expression and speech emotion, the synchronization of lip shape and speech, the rationality of action, the rationality of shot operation, the logical continuity of scene and the consistency of semantic emotion are detected based on the feature set, and a micro video content consistency abnormal feature set is configured.

[0009] In step S300, an AI marketing video special compliance knowledge graph is configured, a semantic path matching is performed by using the multi-modal content fusion feature tensor, a composite violation behavior path is identified, and a structured violation path list and a preliminary compliance evaluation result are generated.

[0010] In step S400, a subjective emotion tendency label set and a marketing intention label sequence are configured, an emotion semantic conflict score value is calculated, a misleading expression segment is identified, an emotion semantic conflict risk label and a structured conflict risk data list are generated.

[0011] In step S500, the multi-class violation identification results are aggregated to form a violation risk feature matrix, the violation behavior trigger source is tracked, and an interpretable violation risk report and a structured modification suggestion list are generated.

[0012] In step S600, a multi-dimensional score dimension and a comprehensive score function are configured, and a quality level label and a quality evaluation report are output.

[0013] As a preferred scheme of the present application, the step S100 is specifically:

[0014] In step S100.1, a multi-modal data original structure set is obtained, and an image semantic feature is extracted.

[0015] The AI marketing video to be audited is accepted by the system, and the AI marketing video comprises data channels of image frame sequences, speech tracks, subtitle texts and background scenes.

[0016] The AI marketing video is input into the video structure decoding module to perform decoding processing, extract time-synchronized image frame sequences, audio tracks and subtitle layers, and mark interframe relationships to form a multi-modal data original structure set.

[0017] The image frame sequence is input into an image semantic feature extraction network. The image semantic feature extraction network adopts a trained multi-scale convolutional neural network structure to analyze the information of the human face, the shape of the commodity, the brand logo pattern, the main and secondary layout of the image, and the dynamic relationship of the action, and performs semantic coding processing on each frame of image to obtain an image semantic feature set.

[0018] Further, a motion analysis operation between the image frame sequence is performed. The image frame sequence difference analysis algorithm, the bidirectional optical flow estimation algorithm, and the key frame judgment algorithm are used to extract the shot motion features, the shot advance rhythm features, and the shot switching density features between the image frame sequence to form a visual rhythm feature set of the image frame sequence.

[0019] Step S100.2, extracting speech expression features based on the multi-modal data original structure set.

[0020] The audio track is input into a speech expression feature extraction network. The speech expression feature extraction network adopts an emotion modeling network based on a deep acoustic modeling structure to extract the speech speed, pitch, tone, speech volatility, and tone type from the human voice and form a speech expression feature set.

[0021] The speech expression feature set and the visual rhythm feature set of the image frame sequence in the image frame sequence are synchronized and aligned to form an audio-visual synchronization association vector structure.

[0022] Step S100.3, extracting text semantic features based on the multi-modal data original structure set.

[0023] Text detection and recognition operations are performed on each frame of image in the subtitle layer. A video text recognition module is called to extract the subtitle content and advertising language description, and a time-sequenced subtitle text sequence corresponding to the frame sequence is output. The subtitle text sequence is input into a text semantic feature extraction network. The text semantic feature extraction network adopts a natural language understanding model based on context semantic modeling to identify the promotion verbs, functional description words, suggestive evaluation words, and risk descriptions, and performs semantic embedding coding on each identified subtitle sentence to extract a text semantic feature set. Further, spatial relationship modeling between the subtitle position and the main object of the picture is performed based on the subtitle recognition to construct a subtitle semantic and image main body linkage relationship graph.

[0024] Step S100.4, extracting scene label information based on the multi-modal data original structure set, and configuring a multi-modal content fusion feature tensor.

[0025] The image frame sequence is input into a scene label extraction network, which is a trained image classification neural network, to perform a frame-by-frame scene recognition operation on the image frame sequence to identify a medical environment, a school environment, a family space, a public passageway, or a commercial selling place, and generate a corresponding scene label information set.

[0026] Further, whether a background brand logo, a competitive product pattern, and a position relationship between a character and a subject object in the scene image exist is identified, and a scene relationship mapping graph is constructed.

[0027] The image semantic feature set, the visual rhythm feature set, the speech expression feature set, the audio-visual synchronization correlation vector structure, the text semantic feature set, the subtitle semantic and image subject linkage relationship graph, the scene label information set, and the scene relationship mapping graph are uniformly encoded, mapped into a unified multi-modal feature space by using a fusion embedding mechanism, and a complete multi-modal content fusion feature tensor structure is constructed.

[0028] As a preferred scheme of the present application, the step S200 specifically includes:

[0029] Step S200.1, detecting a degree of matching between a character facial expression and a speech emotion and a synchronization between a character mouth movement and speech pronunciation based on the image semantic feature set and the speech expression feature set.

[0030] Step S200.2, detecting physical action rationality of a character or an object based on the image frame sequence and detecting rationality of a shot operation strategy based on the visual rhythm feature set.

[0031] Motion trajectory data of the character and the object are extracted based on an image frame sequence difference algorithm, and action trajectory continuity and physical rationality indexes are calculated through an optical flow estimation network and an object posture prediction network, including a speed change, a direction change angle, and a position offset, when the speed change exceeds 500 pixels / second, the direction change angle is greater than 90 degrees, or the position offset of two consecutive frames exceeds 100 pixels without a transition frame, it is marked as an unreasonable physical action abnormality, and the abnormality includes floating, transient, and penetration.

[0032] Lens push rhythm features and shot switching density features in the visual rhythm feature set are obtained, and whether the lens push rhythm features and the shot switching density features conform to human eye visual adaptation rules is analyzed, when a continuous frame segment with a shot switching interval less than 200 milliseconds exceeds 5 seconds, a lens push speed mutation rate exceeds 80%, or a non-natural lens jitter lasts more than 10 frames, it is marked as a shot operation strategy abnormality, and the shot operation strategy abnormality includes non-natural push, lens jump without physical transition, handheld shaking forgery, and non-smooth zooming.

[0033] Step S200.3: Detect the logical continuity of the background scene based on the scene label information set, detect the consistency of semantics and voice emotions based on the text semantic feature set and the voice expression feature set, and configure the micro video content consistency anomaly feature set.

[0034] A frame-sequence scene flow graph is constructed based on the scene label information set to analyze whether the continuous scene labels conform to the actual scene logic. When different logical scenes are switched continuously in a short period of time without any intermediate transition scenes, it is marked as a background scene logic break anomaly. The position continuity of the characters in different scenes is detected based on the scene relationship mapping graph. When it is detected that the position of the characters in different scenes changes instantly and there is a lack of reasonable movement process, it is marked as a character spatial position discontinuity anomaly.

[0035] The semantic type of the subtitle sentence is identified based on the text semantic feature set, and the corresponding voice tone type is identified based on the voice expression feature set. The consistency between the subtitle semantic type and the voice tone type is compared through the semantic tonality mapping table, and the consistency score is calculated. When the consistency score is lower than the threshold of 0.5, it is marked as an anomaly of inconsistency between semantics and voice emotion.

[0036] The anomalies such as inconsistency between facial expressions and voice emotions, asynchrony between lip movements and voice pronunciation, unreasonable physical movements, abnormal camera operation strategies, logical breaks in background scenes, discontinuous spatial positions of characters, and inconsistency between semantics and voice emotions are uniformly coded to generate a set of micro-video content consistency anomaly features, and the corresponding frame sequence positions and anomaly score values ​​are marked.

[0037] As a preferred solution of the present invention, step S300 is specifically as follows:

[0038] Step S300.1. Configure a compliance knowledge graph dedicated to AI marketing videos, match the semantic paths of the compliance knowledge graph based on the multimodal content fusion feature tensor, and obtain a path matching score.

[0039] Step S300.2: Identify the composite violation path based on the path matching score.

[0040] Step S300.3: Generate a compliance assessment tag based on the violation path matching score value, and generate a structured violation path list and a preliminary compliance assessment result based on the compliance assessment tag.

[0041] The system classifies and structures the single violation paths and compound violation paths identified by the semantic path matching engine, and generates a compliance assessment label for each violation path. The system contains five compliance assessment labels, namely:

[0042] Information 1 of the compliance evaluation label: the violation types include: medical efficacy not explicitly stated violation, visual and voice exaggeration violation, and competitor attack violation.

[0043] Information 2 of the compliance evaluation label: the compliance audit semantic node sequence corresponding to the violation path.

[0044] Information 3 of the compliance evaluation label: the multi-modal feature combination type triggered by the violation path, including: image violation feature, text violation feature and voice violation feature.

[0045] Information 4 of the compliance evaluation label: the start frame sequence number and the end frame sequence number corresponding to the violation video segment.

[0046] Information 5 of the compliance evaluation label: the matching score value of the violation path.

[0047] According to the matching score value in the compliance evaluation label and the severity of the violation type, all violation paths and corresponding video segment information are sorted into a structured violation path list, and a preliminary compliance evaluation result of the AI marketing video is generated according to the number of violation paths, violation type and path matching score value.

[0048] As a preferred scheme of the present application, the step S400 specifically comprises:

[0049] Step S400.1, configuring a subjective emotion tendency label set based on a voice expression feature set, and identifying a marketing intention label sequence based on a text semantic feature set.

[0050] Step S400.2, configuring a conflict score function between the emotion tendency label set and the marketing intention label sequence, and obtaining an emotion semantic conflict score value.

[0051] Step S400.3, identifying misleading expression segments according to the emotion semantic conflict score value, and generating an emotion semantic conflict risk label and a structured conflict risk data list.

[0052] The emotion semantic conflict score value calculation is performed on all video frames, and when the emotion semantic conflict score value of any frame is greater than 0.8 , the frame is marked as a high-risk frame of potential misleading expression, and a window sliding detection method is used in the frame sequence time line, if the number of high conflict score frames continuously exceeds 45 frames, the corresponding frame segment is identified as a potential misleading expression segment, and the start and end time information of the current frame segment is output.

[0053] For all identified potential misleading expression segments, an emotion semantic conflict risk label is generated, and the emotion semantic conflict risk label includes: an emotion tendency and marketing intention conflict combination type, a video frame segment start and end frame sequence number, a conflict score maximum value, an average semantic jump rate, a maximum emotion confidence fluctuation value, a conflict level and whether to cover the main sentence expression.

[0054] As a preferred scheme of the present application, the step S500 specifically comprises:

[0055] Step S500.1, aggregate multi-type violation identification results to form a violation risk feature matrix, and track and locate the violation behavior trigger source.

[0056] Step S500.2, label high-risk frame segments according to the calculation results.

[0057] Step S500.3, perform violation behavior causal path tracking and trigger source identification, and generate structured violation modification suggestions and label applicable areas, output explainable violation risk report and structured modification suggestion list.

[0058] For all high-risk video segments, call the violation behavior causal path tracking algorithm, use the violation type, violation label, violation trigger modal type, violation description text and violation feature point frame position index information labeled in the violation risk feature matrix, combine the compliance knowledge graph for path backtracking, extract the violation trigger chain and starting causal point, and output the violation event causal path structure. The violation event causal path structure includes: violation type, first trigger frame sequence number, violation source modal, associated violation content description text, violation feature point frame position index, hit compliance knowledge graph clause node sequence and involved high-risk frame segment start and end frame sequence range.

[0059] Based on the violation type, violation source modal, associated violation content description text and hit compliance knowledge graph clause node sequence extracted in the violation event causal path structure, a structured violation modification suggestion list is constructed. The suggestion includes: suggestion operation type, suggestion content description, suggestion applicable frame segment start and end frame sequence number, involved violation reason text, relied-on compliance knowledge graph clause node number and rule content, and expected violation risk score reduction range.

[0060] The structured violation modification suggestion list is output in JSON structure and table structure respectively.

[0061] Based on the violation risk feature matrix, total violation risk score value and structured violation modification suggestion list, a standard format explainable violation risk report is generated, including: video full-frame violation risk heat map, showing frame-level violation score trend, violation event statistical graph and violation proportion pie chart classified by violation type, all high-risk segment start and end frame sequence index, involved violation type and violation label, violation event causal path visualization graph, structured violation modification suggestion list and total violation risk score value and platform release suggestion.

[0062] The report output is output in PDF format and WebJSON structure respectively.

[0063] As a preferred scheme of the present application, the step S600 specifically comprises:

[0064] Step S600.1, configuring the multi-dimensional scoring dimension of the AI marketing video quality grading evaluation model, and configuring the AI marketing video quality comprehensive score function.

[0065] The AI marketing video quality grading evaluation model is configured, and the AI marketing video quality grading evaluation model performs multi-dimensional quality scoring based on fused multi-source feature information, the input multi-source feature information includes: multi-modal content fusion feature tensor, microscopic video content consistency anomaly feature set, emotional semantic conflict score value, path matching score value and platform propagation feedback index set, the platform propagation feedback index set includes: video complete play rate, like number, comment interaction frequency and content forwarding times four types of core feedback parameters.

[0066] According to the input multi-source feature information, five core scoring dimensions are constructed, which are: video naturalness score item, visual coherence score item, emotional marketing consistency score item, violation risk score item and platform propagation feedback score item, each scoring item adopts a normalized scoring mechanism, and the score range is [0, 1], and a comprehensive score function is introduced to calculate the comprehensive score value.

[0067] Step S600.2, generating a quality level label according to the comprehensive score function result, and outputting a quality evaluation report.

[0068] According to the comprehensive score value, the quality level of the AI marketing video is labeled, and the division standard includes: When the comprehensive score value is greater than 0.7, it is marked as high quality; When the comprehensive score value is greater than 0.4 and less than or equal to 0.7, it is marked as medium quality; When the comprehensive score value is less than or equal to 0.4, it is marked as low quality.

[0069] A structured quality evaluation report is generated, which includes: video quality level label, detailed score value of each scoring item, video frame-level violation risk heat map and marketing emotion and propagation effect analysis summary.

[0070] The structured quality evaluation report is output in WebJSON structure and PDF format respectively.

[0071] For high-quality videos, an automatic green release mark is given, the system audit depth is set to the minimum, and only risk sampling is performed, for medium-quality videos, the system triggers a local enhanced audit process, focusing on detecting risk frame segments and semantic deviation segments, for low-quality videos, the system outputs detailed quality optimization suggestions to the content publisher, and the push process is suspended, and after correction, the score is recalculated.

[0072] Compared with the prior art, the present application has the following advantages:

[0073] 1. By constructing image semantic feature set, visual rhythm feature set, speech expression feature set, text semantic feature set, subtitle semantic and image subject linkage graph, scene label information set and scene relationship mapping graph, and generating unified multi-modal content fusion feature tensor based on fusion embedding mechanism, the synchronization modeling and unified expression of AI marketing video image, speech and text dimension content can be realized without relying on artificial rule setting, which improves the semantic linkage ability and context consistency perception ability among multi-source data, and provides complete data support for subsequent behavior consistency judgment, semantic path matching and video quality scoring.

[0074] 2. By constructing compliance knowledge graph path matching engine, micro video content consistency anomaly detection mechanism and emotional semantic conflict scoring mechanism, multiple types of high-risk violations including inconsistent facial expressions and speech emotions of characters, inconsistent lip movements and speech pronunciation, abnormal lens pushing strategy, conflict between marketing intention and emotional expression, etc. can be identified, and structured violation path list, violation event causal path structure and violation modification suggestion list can be generated, which has traceable, interpretable and visual output ability, realizes the closed loop link from violation identification to rectification suggestion, and improves the intelligent and accurate level of content review.

[0075] 3. By constructing an AI marketing video quality grading evaluation module, the video naturalness score, visual coherence score, emotional and semantic consistency score, compliance path score and platform dissemination feedback index are fused to form a quantitative total quality evaluation score, and accordingly low quality, medium quality and high quality level labels are generated. The system can automatically call different depth review processes or provide content optimization suggestions to the publisher based on the level results, so as to realize the on-demand allocation of review process resources and the adaptive adjustment of risk response strategies, improve the review efficiency while ensuring content compliance and dissemination safety. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 The flowchart of the AI-based marketing video review method provided by the embodiments of the present application is shown. DETAILED DESCRIPTION

[0077] The technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0078] Please refer to Figure 1 , Figure 1 The flowchart of the AI-based marketing video review method provided by the embodiments of the present application is shown.

[0079] In the embodiment, the AI-based marketing video review method can include steps S100, S200, S300, S400, S500 and S600.

[0080] Step S100, a set of multi-modal data original structures is obtained, a set of image semantic features, a set of visual rhythm features, a set of speech expression features, an audio-visual synchronization correlation vector structure, a set of text semantic features, a subtitle semantic and image subject linkage graph, a set of scene label information and a scene relationship mapping graph are extracted, and a multi-modal content fusion feature tensor structure is configured.

[0081] Step S200, the matching degree of facial expression and speech emotion, the synchronization of lip shape and speech, the rationality of action, the rationality of shot operation, the logical continuity of scene and the consistency of semantic emotion are detected based on the feature set, and a set of microscopic video content consistency abnormal features is configured.

[0082] Step S300, an AI marketing video special compliance knowledge graph is configured, a semantic path matching is performed using a multi-modal content fusion feature tensor, a composite violation behavior path is identified, and a structured violation path list and a preliminary compliance evaluation result are generated.

[0083] Step S400, a set of subjective emotional tendency labels and a sequence of marketing intention labels are configured, an emotional semantic conflict score value is calculated, misleading expression segments are identified, an emotional semantic conflict risk label and a structured conflict risk data list are generated.

[0084] Step S500, a plurality of types of violation identification results are aggregated to form a violation risk feature matrix, a violation behavior trigger source is tracked, an interpretable violation risk report and a structured modification suggestion list are generated.

[0085] Step S600, a multi-dimensional score dimension and a comprehensive score function are configured, and a quality level label and a quality evaluation report are output.

[0086] In some specific embodiments, the step S100 specifically includes:

[0087] Step S100.1, a set of multi-modal data original structures is obtained, and image semantic features are extracted.

[0088] The AI marketing video to be reviewed is accepted by the system, and the AI marketing video includes data channels of image frame sequences, speech tracks, subtitle texts and background scenes.

[0089] The AI marketing video is input into a video structure decoding module, decoding processing is performed, time-synchronized image frame sequences, audio tracks and subtitle layers are extracted, inter-frame relationships are labeled, and a set of multi-modal data original structures is constructed.

[0090] The image frame sequence is input into an image semantic feature extraction network, the image semantic feature extraction network adopts a trained multi-scale convolutional neural network structure, analyzes the human face information, commodity shape information, brand logo pattern, image primary and secondary layout and action dynamic relationship, performs semantic coding processing on each frame of image, and obtains an image semantic feature set.

[0091] Further motion analysis operations between the image frame sequence are performed, a difference analysis algorithm, a bidirectional optical flow estimation algorithm and a key frame judgment algorithm are used to extract the shot motion features, shot advance rhythm features and shot switching density features between the image frame sequence, and a visual rhythm feature set of the image frame sequence is constructed.

[0092] Step S100.2, extracting speech expression features based on the multi-modal data original structure set.

[0093] The audio track is input into a speech expression feature extraction network, the speech expression feature extraction network adopts an emotion modeling network based on a deep acoustic modeling structure, extracts speech speed, pitch, tone, speech volatility and tone type from the human voice, and constructs a speech expression feature set.

[0094] The speech expression feature set and the visual rhythm feature set of the image frame sequence in the image frame sequence are synchronized and aligned to form an audio-visual synchronization association vector structure.

[0095] Step S100.3, extracting text semantic features based on the multi-modal data original structure set.

[0096] Text detection and recognition operations are performed on each frame of image in the subtitle layer, a video text recognition module is called to extract the subtitle content and advertising language description, and a time-sequenced subtitle text sequence corresponding to the frame sequence is output. The subtitle text sequence is input into a text semantic feature extraction network, the text semantic feature extraction network adopts a natural language understanding model based on context semantic modeling, identifies promotional verbs, functional description words, suggestive evaluation words and risk descriptions, performs semantic embedding coding on each identified subtitle sentence, extracts a text semantic feature set, and further performs spatial relationship modeling between the subtitle position and the picture main object based on the subtitle recognition, and constructs a subtitle semantic and image main body linkage relationship graph.

[0097] Step S100.4, extracting scene label information based on the multi-modal data original structure set, and configuring a multi-modal content fusion feature tensor.

[0098] The image frame sequence is input into a scene label extraction network, which is a trained image classification neural network, to perform a frame-by-frame scene recognition operation on the image frame sequence to identify and generate a corresponding set of scene label information, including a medical environment, a school environment, a family space, a public passageway, or a commercial selling place.

[0099] Further, whether a background brand logo, a competitive product pattern, and a position relationship between a person and a subject object in the scene image exist are identified, and a scene relationship mapping graph is constructed.

[0100] The image semantic feature set, the visual rhythm feature set, the speech expression feature set, the audio-visual synchronization correlation vector structure, the text semantic feature set, the subtitle semantic and image subject linkage relationship graph, the scene label information set, and the scene relationship mapping graph are uniformly encoded, mapped into a unified multi-modal feature space using a fusion embedding mechanism, and a complete multi-modal content fusion feature tensor structure is constructed.

[0101] In some embodiments, the step S200 specifically includes:

[0102] Step S200.1, detecting a matching degree of a person's facial expression and speech emotion and a synchronization of a person's lip movement and speech pronunciation based on the image semantic feature set and the speech expression feature set.

[0103] The person's facial information in the image semantic feature set is input into a multi-label convolutional emotion recognition network to identify a person's facial expression category, including joy, anger, sadness, surprise, calmness, disgust, and fear, and generate a time-synchronized facial expression label sequence.

[0104] The speech expression feature set is input into a speech emotion recognition network to identify a corresponding speech emotion category and generate a time-synchronized speech emotion label sequence with the facial expression label sequence. By comparing the facial expression label sequence and the speech emotion label sequence, a matching accuracy rate is calculated. When the matching accuracy rate is lower than a threshold value 0.6, it is marked as an inconsistency between the person's facial expression and the speech emotion.

[0105] Based on the image semantic feature set, a person's lip contour and displacement trajectory data are extracted, and a lip movement time sequence is constructed. Based on the speech expression feature set, vowel burst point, consonant transition segment, and rhythm peak data are extracted, and a speech pronunciation time sequence is constructed. The lip movement time sequence and the speech pronunciation time sequence are compared using a dynamic time warping algorithm, and a synchronization deviation rate is calculated. When the synchronization deviation rate is greater than 12 frames, it is marked as an asynchronization between the person's lip movement and the speech pronunciation.

[0106] Step S200.2, detecting the physical action rationality of the person or object based on the image frame sequence, and detecting the rationality of the shot operation strategy based on the visual rhythm feature set.

[0107] Based on the image frame sequence difference algorithm, the motion trajectory data of the person and the object is extracted, and through the optical flow estimation network and the object pose prediction network, the action trajectory continuity and physical rationality indexes are calculated, including: speed change, direction change angle and position offset. When the speed change exceeds 500 pixels / second, the direction change angle is greater than 90 degrees, or the position offset of two consecutive frames exceeds 100 pixels without transition frame, it is marked as physical action unreasonable abnormality. The abnormal phenomena include: floating, transient and penetration.

[0108] The shot advancing rhythm feature and the shot switching density feature in the visual rhythm feature set are obtained, and whether the shot advancing rhythm feature and the shot switching density feature conform to the human eye visual adaptation law is analyzed. When the continuous frame segment with shot switching interval less than 200 milliseconds exceeds 5 seconds, the shot advancing speed mutation rate exceeds 80%, or the non-natural shot jitter lasts more than 10 frames, it is marked as: shot operation strategy abnormality. The shot operation strategy abnormality includes: non-natural advancing, shot jumping without physical transition, handheld shaking forgery and non-smooth zooming.

[0109] Step S200.3, detecting the background scene logical continuity based on the scene label information set, detecting the consistency of semantics and voice emotion based on the text semantic feature set and the voice expression feature set, and configuring the micro video content consistency abnormal feature set.

[0110] Based on the scene label information set, the frame sequence scene flow graph is constructed, and whether the continuous scene label conforms to the actual scene logic is analyzed. When different logical scenes are switched continuously in a short time without intermediate transition scene, it is marked as background scene logic break abnormality. And based on the scene relationship mapping graph, the position continuity of the person in different scenes is detected. When the position of the person in different scenes is detected to change instantaneously and lack of reasonable moving process, it is marked as person spatial position discontinuity abnormality.

[0111] Based on the text semantic feature set, the semantic type of the subtitle sentence is recognized, based on the voice expression feature set, the corresponding voice tone type is recognized, and through the semantic tone mapping table, the consistency of the subtitle semantic type and the voice tone type is compared. The consistency score is calculated. When the consistency score is lower than the threshold value 0.5, it is marked as semantic and voice emotion inconsistency abnormality.

[0112] The character facial expression inconsistency with the voice emotion, the character mouth movement inconsistency with the voice pronunciation, the physical action inconsistency, the abnormality of the lens operation strategy, the abnormality of the background scene logic break, the abnormality of the character space position discontinuity, and the abnormality of the semantic inconsistency with the voice emotion are uniformly coded to generate a micro video content consistency abnormality feature set, and the corresponding frame sequence position and abnormality score value are labeled.

[0113] In some embodiments, the step S300, in particular:

[0114] Step S300.1, configure an AI marketing video compliance knowledge graph, and match the semantic path of the compliance knowledge graph based on the multi-modal content fusion feature tensor, and obtain a path matching score value.

[0115] An AI marketing video compliance knowledge graph is constructed by system configuration, and the AI marketing video compliance knowledge graph is a compliance rule mapping graph based on a graph neural network structure, and the AI marketing video compliance knowledge graph is composed of a plurality of compliance review semantic nodes and a plurality of compliance rule paths between the semantic nodes.

[0116] The compliance review semantic nodes include platform content review standard nodes, advertising law clause nodes, sensitive language definition nodes, competitor attack restriction clause nodes, false advertising judgment nodes, and exemption statement specification nodes, each compliance review semantic node is explicitly associated with a corresponding text keyword, a semantic sentence pattern, an image scene attribute, or a voice expression feature, a plurality of compliance review semantic nodes are connected based on compliance constraint logical relationships, and a complete compliance rule path is formed.

[0117] Based on the multi-modal content fusion feature tensor structure, and using a semantic path matching engine to calculate the path matching score value, the multi-modal content fusion feature tensor structure is input into the semantic path matching engine, the semantic path matching engine is configured based on a graph attention network structure, and the cross-modal semantic path matching task of text, image, and voice features includes three matching dimensions:

[0118] The first dimension: based on the subtitle sentence information extracted from the text semantic feature set and the keyword set of the platform content review standard node and the sensitive language definition node, the semantic matching is performed, and the text compliance path matching result is obtained.

[0119] The second dimension: based on the semantic mapping matching between the scene label information, brand identification patterns, and product form information extracted from the image semantic feature set and the competitor attack restriction clause node and the false advertising judgment node, the image compliance path matching result is obtained.

[0120] The third dimension: Match the intonation change pattern extracted from the speech expression feature set with the speech emotion feature between the sensitive term definition nodes, and obtain the speech compliance path matching result.

[0121] For each compliance rule path, the semantic path matching engine uses the structural attention scoring function to calculate the path matching score value, specifically:

[0122] ;

[0123] Where: The path to be detected , is the number of nodes in the path, For the The fusion features extracted by the node, is the standard eigenvector of the node in the graph, is the graph node attention weight, Represents the similarity function between feature vectors, preferably cosine similarity or inverse normalized Euclidean distance function, Score value for path matching.

[0124] When the path matches the score value , and when the hit node satisfies the composite logical relationship, the path is marked as a high-risk semantic violation path.

[0125] Step S300.2: Identify the composite violation path based on the path matching score.

[0126] The system identifies compound violation paths through the path matching score of the semantic path matching engine and the multimodal feature combination relationship in the multimodal content fusion feature tensor. There are four conditions for compound violation path identification, namely:

[0127] Condition 1: There is a medical environment label in the scene label information of the image semantic feature set, and there is a medical effect description in the subtitle text of the text semantic feature set, and no disclaimer subtitle is detected.

[0128] Condition 2: The speech expression feature set contains obvious exaggerated tone features, and marketing terms related to exaggerated tone are identified in the linkage relationship diagram between subtitle semantics and image body, and no disclaimer subtitles are detected.

[0129] Condition 3: There are competing product images in the image semantic feature set, and there are competitive comparisons or derogatory statements in the linkage relationship diagram between the subtitle semantics and the image body.

[0130] Condition 4: There are celebrity endorsement scene tags in the scene tag information set, and no corresponding non-professional disclaimer subtitles are detected.

[0131] When any one of the four conditions of the composite violation behavior path is met, and the path matching score value exceeds 0.75, the system will mark the corresponding segment of the video as a composite violation behavior.

[0132] Step S300.3, generating a compliance evaluation label based on the violation path matching score value, and generating a structured violation path list and a preliminary compliance evaluation result based on the compliance evaluation label.

[0133] The system classifies and structures the single violation path identified by the semantic path matching engine and the composite violation behavior path, and generates a compliance evaluation label for each violation path, which includes five compliance evaluation labels:

[0134] Information I of the compliance evaluation label: violation type includes: medical efficacy not clearly stated violation, visual and voice exaggeration violation, and competitor attack violation.

[0135] Information II of the compliance evaluation label: the compliance audit semantic node sequence corresponding to the violation path.

[0136] Information III of the compliance evaluation label: the multi-modal feature combination type triggered by the violation path, including: image violation feature, text violation feature and voice violation feature.

[0137] Information IV of the compliance evaluation label: the start frame sequence number and the end frame sequence number corresponding to the violation video segment.

[0138] Information V of the compliance evaluation label: the matching score value of the violation path.

[0139] According to the matching score value and the severity of the violation type in the compliance evaluation label, all violation paths and corresponding video segment information are sorted into a structured violation path list, and the preliminary compliance evaluation result of the AI marketing video is generated according to the number of violation paths, violation types and path matching score values. The preliminary compliance evaluation result includes:

[0140] Level one: compliant, no violation path is identified;

[0141] Level two: mild violation, only one violation path with a matching score value less than 0.75 is identified;

[0142] Level three: moderate violation, two or more violation paths with a matching score value between 0.75 and 0.9 are identified;

[0143] Level four: severe violation, any one violation path with a matching score value exceeding 0.9 or any one composite violation behavior path is identified.

[0144] In some embodiments, the step S400 specifically includes:

[0145] Step S400.1, configure a subjective emotion tendency label set based on the voice expression feature set, and identify a marketing intention label sequence based on the text semantic feature set.

[0146] The speech rate, tone, pitch, speech volatility and tone type extracted from the voice expression feature set are input to a multi-channel emotion recognition neural network, which adopts an acoustic feature and context attention fusion mechanism. The human face information in the image semantic feature set is input to a multi-label convolutional emotion recognition network, which performs facial expression classification on each frame. The output results of the multi-channel emotion recognition neural network and the multi-label convolutional emotion recognition network are fused through an emotion consistency fusion strategy to generate a time-synchronized subjective emotion tendency label set.

[0147] The subjective emotion tendency label set includes joy, exuberance, calmness, neutrality, melancholy, sadness, anger and anxiety. Each frame of video corresponds to an emotion tendency label and a confidence score.

[0148] The frame-synchronized caption sentences in the text semantic feature set are input to a marketing intention recognition network, which is a semantic intention classification model based on a bidirectional context attention mechanism. The marketing intention recognition network outputs the marketing intention label corresponding to each caption sentence according to the sentence pattern, keyword weight and semantic directionality. The marketing intention label includes product recommendation type, interest-driven type, emotion infection type, professional endorsement type and use scenario guidance type. The marketing intention label corresponds to the video frame sequence timeline and forms a marketing semantic orientation structure of the video.

[0149] Step S400.2, configure a conflict scoring function between the emotion tendency label set and the marketing intention label sequence, and obtain an emotion semantic conflict score value.

[0150] Based on the subjective emotion tendency label set and the marketing intention label sequence, four typical conflict combination criteria are set, which are:

[0151] Combination one: when the subjective emotion tendency label is sadness or melancholy, and the marketing intention label is interest-driven type or emotion infection type, it is defined as high-level conflict.

[0152] Combination two: when the subjective emotion tendency label is anger or anxiety, and the marketing intention label is use scenario guidance type or professional endorsement type, it is defined as high-level conflict.

[0153] Combination three: when the subjective emotion tendency label is calmness or neutrality, and the marketing intention label is emotion infection type, it is defined as medium-level conflict.

[0154] Combination 4: When the subjective emotion tendency label matches the marketing intention label, it is defined as non-conflict.

[0155] Define the sentiment semantic conflict scoring function as follows:

[0156] ;

[0157] Where: For the The emotional semantic conflict score of the frame, is the confidence change rate of emotional tendency, , is the semantic change rate of the marketing intention label, and its value is , Subjective emotional tendency label Tags with marketing intent The conflict level scores are 0 no conflict, 0.5 medium conflict, and 1 high conflict. is the weighting coefficient, and the preferred value is , the sum of the three is 1.

[0158] Step S400.3: Identify misleading expression segments based on the emotion-semantic conflict score, and generate an emotion-semantic conflict risk label and a structured conflict risk data list.

[0159] The emotional semantic conflict score value calculation is performed on the entire video frame. When the emotional semantic conflict score value of any frame , it is marked as a high-risk frame for potential misleading expression, and a window sliding detection method is used in the frame sequence timeline. If the number of high conflict score frames exceeds 45 frames continuously, the corresponding frame segment is identified as a potential misleading expression segment, and the start and end time information of the current frame segment is output.

[0160] For all segments identified as potentially misleading expressions, an emotional semantic conflict risk label is generated. The emotional semantic conflict risk label includes: the combination type of conflict between emotional tendency and marketing intention, the start and end frame numbers of the video segment, the maximum conflict score, the average semantic jump rate, the maximum emotional confidence fluctuation value, the conflict level, and whether it covers the main sentence expression.

[0161] In some specific implementations, the step S500 is specifically:

[0162] Step S500.1: Aggregate multiple types of violation identification results to form a violation risk feature matrix, and track the violation causal path and locate the violation trigger source.

[0163] The micro-video content consistency anomaly feature set, structured violation path list and emotional semantic conflict risk label are time-synchronized and feature-merged to form a frame-level violation risk feature matrix. Each row of the violation risk feature matrix contains: video frame number, violation risk type, violation risk label, violation score value, risk level, violation reason text, violation trigger mode type and violation feature point frame index.

[0164] Step S500.2: Mark high-risk frame segments according to the calculation results.

[0165] For each video frame, the total violation risk score is calculated by weighting the micro-video content consistency anomaly feature set, compliance knowledge graph, and emotional semantic conflict score in the violation risk feature matrix. Specifically, the total violation risk score is:

[0166] ;

[0167] Where: It is The total violation risk score value of the frame, It is a collection of abnormal features of microscopic video content consistency, It is a compliance knowledge graph. is the emotional semantic conflict score, 、 、 is the weighting coefficient, and its value is 、 、 .

[0168] when When , it is marked as a high-risk frame. When , it is marked as a medium-risk frame, and the rest are low-risk frames. Continuous high-risk frame segments are merged into high-risk video segments, and the start and end frame sequence indexes are recorded.

[0169] Step S500.3: Perform causal path tracing of violation behaviors and identification of trigger sources, generate structured violation modification suggestions and mark applicable areas, and output an explainable violation risk report and a structured modification suggestion list.

[0170] For all high-risk video clips, the violation causal path tracing algorithm is called to use the violation type, violation label, violation trigger mode type, violation description text and violation feature point frame index information marked in the violation risk feature matrix, combined with the compliance knowledge graph to perform path backtracking, extract the violation trigger chain and the starting causal point, and output the violation event causal path structure. The violation event causal path structure includes: violation type, first trigger frame number, violation source mode, associated violation content description text, violation feature point frame index, hit compliance knowledge graph clause node sequence and the start and end frame sequence range of the high-risk frame segment involved.

[0171] Based on the extracted violation type, violation source mode, associated violation content description text and hit compliance knowledge graph clause node sequence in the violation event causal path structure, a structured violation modification suggestion list is constructed, including: suggestion operation type, suggestion content description, suggestion applicable frame segment start and end frame sequence number, involved violation reason text, based on compliance knowledge graph clause node number and rule content, and expected violation risk score reduction range.

[0172] The structured violation modification suggestion list is output in JSON structure and table structure respectively.

[0173] Based on the violation risk feature matrix, total violation risk score value and structured violation modification suggestion list, a standard format interpretable violation risk report is generated, including: video full frame violation risk heat map, showing frame level violation score trend, violation event statistics graph and violation proportion pie chart according to violation type classification, all high risk segment start and end frame sequence index, involved violation type and violation label, violation event causal path visualization graph, structured violation modification suggestion list and total violation risk score value and platform release suggestion.

[0174] The report output is output in PDF format and WebJSON structure respectively.

[0175] In some embodiments, the step S600, in particular:

[0176] Step S600.1, configure the multi-dimensional score dimension of the AI marketing video quality grading evaluation model, and configure the AI marketing video quality comprehensive score function.

[0177] The AI marketing video quality grading evaluation model is configured. The AI marketing video quality grading evaluation model performs multi-dimensional quality scoring based on the fusion of input multi-source feature information. The input multi-source feature information includes: multi-modal content fusion feature tensor, microscopic video content consistency anomaly feature set, emotional semantic conflict score value, path matching score value and platform propagation feedback index set, which includes: video complete play rate, like number, comment interaction frequency and content forwarding times four types of core feedback parameters.

[0178] According to the input multi-source feature information, five core score dimensions are constructed, which are: video naturalness score item, visual coherence score item, emotional marketing consistency score item, violation risk score item and platform propagation feedback score item.

[0179] Each score item adopts a normalized scoring mechanism, with a score range of [0, 1], and a comprehensive score function is introduced to calculate the comprehensive score value.

[0180] The video quality comprehensive score function is set as:

[0181]

[0182] wherein: is the comprehensive quality score value of the AI marketing video, is the video naturalness score value, is the visual coherence score value, is the emotional marketing consistency score value, is the rule violation risk score value, is the platform propagation feedback score value, is the weighting coefficient corresponding to each score item, and is configured as and 0.2, satisfying the sum of the weighting coefficients being 1.

[0183] Step S600.2. Generate a quality level label according to the result of the comprehensive score function, and output a quality evaluation report.

[0184] According to the comprehensive score value, the quality level of the AI marketing video is labeled, and the division standards include: when , it is marked as high quality, when , it is marked as medium quality, and when , it is marked as low quality.

[0185] A structured quality evaluation report is generated, including: video quality level label, detailed score value of each score item, video frame-level rule violation risk heat map, and marketing emotion and propagation effect analysis summary.

[0186] The structured quality evaluation report is output in WebJSON structure and PDF format respectively.

[0187] For high-quality videos, an automatic green release mark is given, the system audit depth is set to the minimum, and only risk sampling is performed. For medium-quality videos, the system triggers a local enhanced audit process, focusing on detecting risk frame segments and semantic deviation segments. For low-quality videos, the system outputs detailed quality optimization suggestions to the content publisher, and temporarily suspends the push process until the revisions are made and re-scored.

[0188] ​In the above, in actual application, first, after receiving the AI marketing video to be audited, the multi-modal video content analysis model deployed on the audit server is called to analyze the video frame by frame, extract image frame sequence, voice track, subtitle text and background scene image, for image frame sequence, extract static visual information, character action feature and lens motion parameter, for voice track, extract tone, speed and pronunciation consistency feature, for subtitle text, extract text semantic embedding vector and sentiment tendency label, for background image, extract scene category and composition element, the above features are encoded into fused multi-modal content expression tensor through unified feature mapping channel after extraction, as the input basis for subsequent content consistency detection and violation behavior analysis.

[0189] After the multi-modal feature extraction is completed, based on the principle of natural continuity of video and human viewing habits, a six-dimensional perception consistency detection mechanism is constructed, which includes: expression and tone matching degree detection, lip shape and pronunciation synchronization degree detection, action and physical law consistency detection, lens push strategy non-natural motion detection, background scene logical coherence analysis, and expression consistency judgment between emotional expression and marketing semantics, the consistency score vector is generated by rule coding and abnormal score algorithm, the unified standard is floating point score between 0 and 1, the dimension score below 0.3 is judged as significant abnormality, which provides basic data support for subsequent violation path analysis and risk score.

[0190] Then, based on the preset AI marketing video compliance knowledge graph, the knowledge graph nodes include platform rule clauses, corresponding articles of advertising law, common sensitive expression library, industry constraint expression and false propaganda criteria, etc., the semantic features in the fused content expression tensor are matched by vector, the potential violation path is identified, the matching method adopts multi-hop path search and semantic attention weighting mechanism, which can judge the complex violation structure of cross-modal synthesis, such as excessive emotional incitement tone matching with efficacy description, etc., finally output the compliance preliminary judgment label, and generate graph path matching record, which is used for violation content causal path reconstruction and risk positioning.

[0191] Then, the tone, speed and facial expression features extracted from the video are input into the trained emotion recognition network to generate standard emotion category labels, combined with the text semantic recognition module, judge the marketing purpose type, including emotional appeal type, interest stimulation type, rational exposition type, etc., compare the output results of the two, generate emotion semantic consistency score according to the preset conflict judgment matrix, if there is obvious conflict expression such as "sad expression matching enthusiastic promotion language", and the score is lower than 0.4, it is automatically marked as high-risk conflict content, which provides an important reference path for subsequent violation content judgment.

[0192] Secondly, the consistency detection result, compliance evaluation label and emotional semantic conflict score result are aggregated to construct a frame-level unit violation risk score matrix. A violation cause path tracking algorithm is used to locate specific violation segments. Four elements of violation trigger source content, violation reason, graph path node and violation type are output. According to different types of violation content, a preset structured suggestion template library is called to output operation instructions including word replacement suggestion, tone adjustment suggestion, shot switching scheme and subtitle modification scheme. Finally, an explainable and executable audit report is generated. The report marks the violation segment timestamp and is accompanied by the modification threshold condition corresponding to each type of suggestion, for example, the word content is forced to modify when the violation risk score is greater than 0.7.

[0193] Finally, the fused multi-modal content expression tensor, consistency score vector, violation risk score matrix and graph matching result are input into a quality evaluation model to generate a comprehensive score of the marketing video. The evaluation model outputs the video quality level, which is divided into three levels: low quality score <0.4, medium quality 0.4≤score <0.75 and high quality score ≥0.75. Different audit response strategies are selected according to the quality level, for example, low-quality videos trigger a re-audit process, high-quality videos are automatically pushed to a pre-release platform, and optimization processes are triggered according to the suggestion content. The entire processing flow forms a closed-loop execution logic from data analysis, compliance audit, risk positioning to optimization feedback.

[0194] Although embodiments of the present application have been shown and described, it will be understood by those having ordinary skill in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. The AI-based marketing video review method is characterized by: The steps include: S100, obtaining a set of original multimodal data structures, extracting an image semantic feature set, a visual rhythm feature set, a speech expression feature set, an audio-visual synchronization association vector structure, a text semantic feature set, a subtitle semantics and image subject linkage relationship diagram, a scene label information set, and a scene relationship mapping diagram, and configuring a multimodal content fusion feature tensor structure; S200, based on the feature set, detecting the matching degree between the character's expression and voice emotion, the lip shape and voice synchronization, the rationality of the action, the rationality of the camera movement, the logical continuity of the scene, and the semantic emotion consistency, and configuring the micro-video content consistency abnormality feature set; S300, configures a compliance knowledge graph dedicated to AI marketing videos, uses multimodal content fusion feature tensors for semantic path matching, identifies complex violation paths, and generates a structured violation path list and preliminary compliance assessment results; S400: Configure a subjective emotion tendency label set and a marketing intention label sequence, calculate an emotion semantic conflict score, identify misleading expression segments, and generate an emotion semantic conflict risk label and a structured conflict risk data list; S500: Aggregate multiple violation identification results to form a violation risk feature matrix, track the triggering sources of violations, and generate an explainable violation risk report and a structured modification suggestion list; S600: Configure multi-dimensional scoring dimensions and comprehensive scoring functions, and output quality grade labels and quality assessment reports.

2. The AI-based marketing video review method according to claim 1, characterized in that: The S100 is specifically: S100.

1. Obtain a set of original structures of multimodal data and extract image semantic features; The system accepts an AI marketing video for review, wherein the AI ​​marketing video includes: image frame sequences, voice tracks, subtitle text, and background scene data channels; Input the AI ​​marketing video into the video structure decoding module, perform decoding processing, extract the time-synchronized image frame sequence, audio track and subtitle layer, and annotate the relationship between frames to form a multimodal data original structure set; The image frame sequence is input into the image semantic feature extraction network. The image semantic feature extraction network uses a trained multi-scale convolutional neural network structure to analyze the facial information of the characters, the shape of the products, the brand logo pattern, the primary and secondary layout of the image, and the dynamic relationship of the action. It performs semantic encoding processing on each frame of the image to obtain a set of image semantic features. Furthermore, the motion analysis operation between image frame sequences is performed, and the lens motion features, lens advancement rhythm features and lens switching density features between image frame sequences are extracted using the image frame sequence difference analysis algorithm, the bidirectional optical flow estimation algorithm and the key frame judgment algorithm, thereby forming a visual rhythm feature set of the image frame sequence. S100.

2. Extracting speech expression features based on the original structure set of multimodal data; The audio track is fed into a speech expression feature extraction network. This network uses an emotion modeling network based on a deep acoustic modeling structure to extract speech rate, pitch, tone, voice volatility, and tone type from the human voice, and forms a speech expression feature set. Synchronously aligning the speech expression feature set with the visual rhythm feature set of the image frame sequence in the image frame sequence to form an audio-visual synchronization association vector structure; S100.

3. Extracting text semantic features based on the original structure set of multimodal data; Perform text detection and recognition operations on each frame of the subtitle layer, call the video text recognition module to extract the subtitle content and advertising description, and output a time-sequenced subtitle text sequence corresponding to the frame sequence. The subtitle text sequence is input into the text semantic feature extraction network. The text semantic feature extraction network adopts a natural language understanding model based on contextual semantic modeling to identify promotional verbs, functional descriptive words, suggestive evaluation words and risk descriptions, and performs semantic embedding coding on each identified subtitle sentence to extract a set of text semantic features. Furthermore, based on subtitle recognition, the spatial relationship between the subtitle position and the main object of the picture is modeled to construct a linkage relationship diagram between the subtitle semantics and the main image. S100.

4. Extract scene label information based on the original structure set of multimodal data, and configure a multimodal content fusion feature tensor; The image frame sequence is input into a scene label extraction network. The scene label extraction network is a trained image classification neural network that performs frame-by-frame scene recognition on the image frame sequence. The recognition includes: medical environment, school environment, home space, public passage or commercial store, and generates a corresponding scene label information set; Further identify whether there are background brand logo patterns, competing product patterns, and the positional relationship between people and main objects in the scene in the scene image, and construct a scene relationship map; The image semantic feature set, visual rhythm feature set, speech expression feature set, audio-visual synchronization association vector structure, text semantic feature set, subtitle semantics and image subject linkage relationship diagram, scene label information set and scene relationship mapping diagram are uniformly encoded and mapped into a unified multimodal feature space using a fusion embedding mechanism, and a complete multimodal content fusion feature tensor structure is constructed.

3. The AI-based marketing video review method according to claim 1, characterized in that: The S200 is specifically: S200.

1. Detect the degree of matching between facial expressions and speech emotions, and the synchronization between lip movements and speech pronunciation, based on the image semantic feature set and the speech expression feature set; The facial information of the characters in the image semantic feature set is input into a multi-label convolutional emotion recognition network to identify the facial expression categories of the characters, including joy, anger, sadness, surprise, calmness, disgust and fear, and generate a time-synchronized sequence of facial expression labels; The speech expression feature set is input into the speech emotion recognition network to identify the corresponding speech emotion category and generate a speech emotion label sequence that is time-synchronized with the facial expression label sequence. The facial expression label sequence and the speech emotion label sequence are compared to calculate the matching accuracy. When the matching accuracy falls below a threshold of 0.6, it is marked as an inconsistency between the facial expression and speech emotion of the person. Based on the image semantic feature set, the character's lip shape contour and displacement trajectory data are extracted, and a lip movement time series is constructed. Based on the speech expression feature set, the vowel burst point, consonant transition segment and rhythm peak data are extracted to construct a speech pronunciation time series. The dynamic time warping algorithm is used to compare the lip movement time series with the speech pronunciation time series, and the synchronization offset rate is calculated. When the synchronization offset rate is greater than 12 frames, it is marked as an abnormal synchronization between the character's lip movement and speech pronunciation. S200.

2. Detecting the rationality of the physical movements of people or objects based on the image frame sequence, and detecting the rationality of the camera movement strategy based on the visual rhythm feature set; Using an image frame sequence difference algorithm to extract motion trajectory data for people and objects, the optical flow estimation network and the object pose prediction network are used to calculate trajectory continuity and physical plausibility indicators, including speed change, direction change angle, and position offset. When the speed change exceeds 500 pixels / second, the direction change angle is greater than 90 degrees, or the position offset between two consecutive frames exceeds 100 pixels with no transition frames, it is marked as an unreasonable physical motion anomaly. Abnormal phenomena include floating, transients, and penetration. Obtain the shot advancement rhythm features and shot switching density features from the visual rhythm feature set and analyze whether they conform to the laws of human visual adaptation. If the continuous frame segments with a shot switching interval of less than 200 milliseconds exceed 5 seconds, the shot advancement speed mutation rate exceeds 80%, or unnatural camera shake persists for more than 10 frames, it is marked as abnormal camera operation strategy. Abnormal camera operation strategies include unnatural advancement, shot jumps without physical transitions, artificial handheld shaking, and non-smooth zooming. S200.

3. Detecting the logical continuity of the background scene based on the scene label information set, detecting the consistency of semantics and speech emotions based on the text semantic feature set and the speech expression feature set, and configuring a micro-video content consistency anomaly feature set; Based on the scene label information set, a frame sequence scene flow graph is constructed to analyze whether the continuous scene labels conform to the actual scene logic. When different logical scenes are switched continuously in a short period of time without intermediate transition scenes, it is marked as a background scene logic break anomaly. The position continuity of the characters in different scenes is detected based on the scene relationship map. When the position of the characters in different scenes is detected to change instantly and there is no reasonable movement process, it is marked as a character spatial position discontinuity anomaly. The semantic type of the subtitle sentence is identified based on the text semantic feature set, and the corresponding voice tone type is identified based on the voice expression feature set. The consistency between the subtitle semantic type and the voice tone type is compared using the semantic tonality mapping table, and a consistency score is calculated. When the consistency score falls below the threshold of 0.5, it is marked as an anomaly of inconsistency between semantics and voice emotion; The anomalies such as inconsistency between facial expressions and voice emotions, asynchrony between lip movements and voice pronunciation, unreasonable physical movements, abnormal camera operation strategies, logical breaks in background scenes, discontinuous spatial positions of characters, and inconsistency between semantics and voice emotions are uniformly coded to generate a set of micro-video content consistency anomaly features, and the corresponding frame sequence positions and anomaly score values ​​are marked.

4. The AI-based marketing video review method according to claim 1, characterized in that: The S300 is specifically as follows: S300.

1. Configure a compliance knowledge graph dedicated to AI marketing videos, match semantic paths in the compliance knowledge graph based on multimodal content fusion feature tensors, and derive a path matching score. Constructing a compliance knowledge graph dedicated to AI marketing videos through system configuration. The compliance knowledge graph dedicated to AI marketing videos is: a compliance rule mapping graph based on a graph neural network structure. The compliance knowledge graph dedicated to AI marketing videos is composed of multiple compliance review semantic nodes and compliance rule paths between multiple semantic nodes; The compliance review semantic nodes include: platform content review standard node, advertising law clause node, sensitive term definition node, competitor attack restriction clause node, false advertising determination node, and disclaimer specification node. Each compliance review semantic node is clearly associated with a corresponding text keyword, semantic sentence pattern, image scene attribute, or voice expression feature. Multiple compliance review semantic nodes are connected based on compliance constraint logical relationships to form a complete compliance rule path. Based on the multimodal content fusion feature tensor structure, the semantic path matching engine is used to calculate the path matching score. The multimodal content fusion feature tensor structure is input into the semantic path matching engine. The semantic path matching engine is configured based on the graph attention network structure. The cross-modal semantic path matching task of text, image and speech features includes three matching dimensions: The first dimension: semantic matching is performed based on the subtitle sentence information extracted from the text semantic feature set and the keyword set of the platform content review standard node and sensitive term definition node, and the text compliance path matching result is obtained; The second dimension: Based on the semantic mapping between the scene label information, brand logo pattern and product form information extracted from the image semantic feature set and the competitor attack restriction clause node and false advertising determination node, the image compliance path matching result is obtained; The third dimension: Matching the intonation variation patterns extracted from the speech expression feature set with the speech emotion features between the sensitive term definition nodes, and obtaining the speech compliance path matching results; For each compliance rule path, the semantic path matching engine uses the structural attention scoring function to calculate the path matching score value, specifically: ； Where: The path to be detected , is the number of nodes in the path, For the The fusion features extracted by the node, is the standard eigenvector of the node in the graph, is the graph node attention weight, Represents the similarity function between feature vectors, expressed as cosine similarity or reciprocal normalized Euclidean distance function, Score value for path matching; When the path matches the score value , and when the hit node satisfies the composite logical relationship, the path is marked as a high-risk semantic violation path; S300.

2. Identify the path of compound violation behavior based on the path matching score; The system identifies compound violation paths through the path matching score of the semantic path matching engine and the multimodal feature combination relationship in the multimodal content fusion feature tensor. There are four conditions for compound violation path identification, namely: Condition 1: The scene label information of the image semantic feature set contains a medical environment label, and the subtitle text of the text semantic feature set contains a description of the medical effect, and no disclaimer subtitle is detected; Condition 2: The speech expression feature set contains obvious exaggerated tone features, and the subtitle semantics and image body linkage relationship diagram identifies marketing terms related to exaggerated tone, and no disclaimer subtitles are detected; Condition 3: The image semantic feature set contains images of competing products, and the caption semantics and image subject linkage diagram contains competitive comparisons or derogatory statements; Condition 4: There are celebrity endorsement scene tags in the scene tag information set, and no corresponding non-professional disclaimer subtitles are detected; When the composite violation path meets any of the four conditions and the path matching score exceeds 0.75, the system will mark the corresponding video segment as a composite violation; S300.

3. Generate a compliance assessment tag based on the violation path matching score value, and generate a structured violation path list and preliminary compliance assessment results based on the compliance assessment tag; The system classifies and structures the single violation paths and compound violation paths identified by the semantic path matching engine, and generates a compliance assessment label for each violation path. The system contains five compliance assessment labels, namely: Information on the Compliance Assessment Label:

1. Violation types include: failure to clearly state medical efficacy, visual and audio exaggeration, and competitor attack. Information 2 of the compliance assessment label: the compliance review semantic node sequence corresponding to the violation path; Compliance assessment label information 3: The type of multimodal feature combination triggered by the violation path, including image violation features, text violation features, and voice violation features; Information 4 of the compliance assessment label: the starting frame number and ending frame number of the offending video clip; Information 5 of the compliance assessment tag: the matching score value of the violation path; Based on the matching score value and violation type severity in the compliance assessment tag, all violation paths and corresponding video clip information are organized into a structured violation path list. Based on the number of violation paths, violation type, and path matching score value, a preliminary compliance assessment result for the AI ​​marketing video is generated. The preliminary compliance assessment result includes: Level 1: Compliant, no illegal paths identified; Level 2: Minor violation, only one violation path with a matching score lower than 0.75 is identified; Level 3: Moderate violation, two or more violation paths with matching scores between 0.75 and 0.9 are identified; Level 4: Severe violation, identifying any violation path with a matching score value exceeding 0.9 or any compound violation path.

5. The AI-based marketing video review method according to claim 1, characterized in that: The S400 is specifically: S400.

1. Configure a subjective emotion tendency label set based on the speech expression feature set, and identify a marketing intention label sequence based on the text semantic feature set; The speech rate, tone, pitch, voice volatility, and tone type extracted from the speech expression feature set are input into a multi-channel emotion recognition neural network. The multi-channel emotion recognition neural network uses an acoustic feature and contextual attention fusion mechanism. The facial information of the characters in the image semantic feature set is input into a multi-label convolutional emotion recognition network. The multi-label convolutional emotion recognition network performs facial expression classification on each frame. The output results of the multi-channel emotion recognition neural network are fused with the output results of the multi-label convolutional emotion recognition network through an emotion consistency fusion strategy to generate a time-synchronized set of subjective emotion tendency labels. The subjective emotion tendency label set includes: joy, elation, calmness, neutrality, depression, sadness, anger, and anxiety. Each frame of the video corresponds to an emotion tendency label and confidence score. The frame-synchronized subtitle sentences in the text semantic feature set are input into the marketing intent recognition network. The marketing intent recognition network is a semantic intent classification model based on a bidirectional contextual attention mechanism. The marketing intent recognition network outputs the marketing intent label corresponding to each frame subtitle sentence based on sentence structure, keyword weight, and semantic directionality. Marketing intent labels include: product recommendation, profit-driven, emotional influence, professional endorsement, and usage scenario guidance. The marketing intent labels correspond to the video frame timeline and form a marketing semantic guidance structure for the video. S400.

2. Configure a conflict scoring function between the emotion tendency tag set and the marketing intention tag sequence, and obtain an emotion semantic conflict score value; Based on the subjective emotion tendency label set and marketing intention label sequence, four typical conflict judgment combinations are set, namely: Combination 1: When the subjective emotional tendency label is sadness or depression, and the marketing intention label is profit-driven or emotional contagion, it is defined as high-level conflict; Combination 2: When the subjective emotion tendency label is anger or anxiety, and the marketing intention label is usage scenario guidance or professional endorsement, it is defined as high-level conflict; Combination 3: When the subjective emotional tendency label is calm or neutral, and the marketing intention label is emotional contagion, it is defined as medium-level conflict; Combination 4: When the subjective emotion tendency label matches the marketing intention label, it is defined as no conflict; Define the sentiment semantic conflict scoring function as follows: ； Where: For the The emotional semantic conflict score of the frame, is the confidence change rate of emotional tendency, , is the semantic change rate of the marketing intention label, and its value is , Label for subjective emotional tendency Tags with marketing intent The conflict level scores are 0 no conflict, 0.5 medium conflict, and 1 high conflict. is the weighting coefficient, and the setting value is , the sum of the three is 1; S400.

3. Identify misleading expression segments based on the sentiment-semantic conflict score, and generate sentiment-semantic conflict risk labels and a structured conflict risk data list; The emotional semantic conflict score value calculation is performed on the entire video frame. When the emotional semantic conflict score value of any frame , it is marked as a high-risk frame for potential misleading expression, and a window sliding detection method is used in the frame sequence timeline. If the number of high conflict score frames exceeds 45 frames in a row, the corresponding frame segment is identified as a potential misleading expression segment, and the start and end time information of the current frame segment is output; For all segments identified as potentially misleading expressions, an emotional semantic conflict risk label is generated. The emotional semantic conflict risk label includes: the combination type of conflict between emotional tendency and marketing intention, the start and end frame numbers of the video segment, the maximum conflict score, the average semantic jump rate, the maximum emotional confidence fluctuation value, the conflict level, and whether it covers the main sentence expression.

6. The AI-based marketing video review method according to claim 1, characterized in that: The S500 is specifically: S500.

1. Aggregate multiple violation identification results to form a violation risk feature matrix, and then trace the causal path of violations and locate the triggering source of the violation; The micro-video content consistency anomaly feature set, structured violation path list, and emotional semantic conflict risk label are time-series synchronized and feature-merged to form a frame-level violation risk feature matrix. Each row of the violation risk feature matrix contains: video frame number, violation risk type, violation risk label, violation score value, risk level, violation reason text, violation trigger mode type, and violation feature point frame index; S500.

2. Mark high-risk frame segments based on the calculation results; For each video frame, the total violation risk score is calculated by weighting the micro-video content consistency anomaly feature set, compliance knowledge graph, and emotional semantic conflict score in the violation risk feature matrix. Specifically, the total violation risk score is: ； Where: It is The total violation risk score value of the frame, is a collection of abnormal features of microscopic video content consistency, It is a compliance knowledge graph. is the emotional semantic conflict score, 、 、 is the weighting coefficient, and its value is 、 、 ; when When , it is marked as a high-risk frame. When , it is marked as a medium-risk frame, and the rest are low-risk frames. Continuous high-risk frame segments are merged into high-risk video segments, and the start and end frame sequence indexes are recorded; S500.

3. Perform causal path tracing and trigger source identification for violations, generate structured violation modification suggestions and mark applicable areas, and output an explainable violation risk report and a list of structured modification suggestions; For all high-risk video clips, the violation causal path tracing algorithm is called. The violation type, violation label, violation trigger mode type, violation description text and violation feature point frame index information marked in the violation risk feature matrix are used in combination with the compliance knowledge graph to perform path backtracking, extract the violation trigger chain and the starting causal point, and output the violation event causal path structure. The violation event causal path structure includes: violation type, first trigger frame number, violation source mode, associated violation content description text, violation feature point frame index, hit compliance knowledge graph clause node sequence and the start and end frame sequence range of the high-risk frame segment involved; Based on the violation type, violation source modality, associated violation content description text, and the hit compliance knowledge graph clause node sequence extracted from the causal path structure of the violation event, a structured violation modification suggestion list is constructed. The suggestion includes: the recommended operation type, the recommended content description, the recommended applicable frame segment start and end frame sequence, the violation reason text involved, the compliance knowledge graph clause node number and rule content based on it, and the expected violation risk score reduction range; The structured violation modification suggestion list is output in JSON structure and table structure respectively; Based on the violation risk feature matrix, total violation risk score, and structured violation modification suggestion list, a standard format, explainable violation risk report is generated. This report includes: a full-frame violation risk heat map showing the trend of frame-level violation score changes, a statistical chart of violation events classified by violation type and a pie chart of violation ratios, an index of the start and end frames of all high-risk clips, the violation types and violation labels involved, a visualization of the causal path of violation events, a structured violation modification suggestion list, a total violation risk score, and platform release suggestions; The report is output in PDF format and WebJSON structure respectively.

7. The AI-based marketing video review method according to claim 1, characterized in that: The S600 is specifically: S600.

1. Configure the multi-dimensional scoring dimensions of the AI ​​marketing video quality grading assessment model and the comprehensive scoring function for AI marketing video quality. Configure an AI marketing video quality grading and assessment model. This model performs multi-dimensional quality scoring based on the fusion of multi-source feature information. The input multi-source feature information includes: a multimodal content fusion feature tensor, a micro-video content consistency anomaly feature set, an emotional semantic conflict score, a path matching score, and a platform communication feedback indicator set. The platform communication feedback indicator set includes four core feedback parameters: video completion rate, number of likes, comment interaction frequency, and content forwarding count. Five core scoring dimensions are constructed based on the input multi-source feature information, namely: video naturalness scoring, visual coherence scoring, emotional marketing consistency scoring, violation risk scoring, and platform communication feedback scoring. Each scoring item adopts a normalized scoring mechanism with a score range of [0,1], and introduces a comprehensive scoring function to calculate the comprehensive score value; S600.

2. Generate a quality grade label based on the comprehensive scoring function results and output a quality assessment report; The quality level of AI marketing videos is marked based on the comprehensive score. The classification criteria include: When , it is marked as high quality, when When , it is marked as medium quality, when When , it is marked as low quality; Generate a structured quality assessment report, which includes: video quality grade labels, detailed rating values ​​for each rating item, a heat map of video frame-level violation risks, and a summary of marketing sentiment and communication effect analysis; The structured quality assessment report will be output in WebJSON structure and PDF format respectively; For high-quality videos, a green release mark is automatically assigned, the system review depth is set to the minimum, and only risk spot checks are performed. For medium-quality videos, the system triggers a local enhanced review process, focusing on detecting risky frames and semantically deviated segments. For low-quality videos, the system outputs detailed quality optimization suggestions to the content publisher and temporarily suspends the push process until corrections are made and the score is re-scored.

Images