A video auditing method and device, electronic equipment and storage medium

Abstract

The application provides a video auditing method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a video to be audited and specified data associated with the video; obtaining a set of should-be constraints, a set of real observation and a set of evidence nodes according to the video to be audited and the specified data; writing the set of should-be constraints, the set of real observation and the set of evidence nodes into an auditing graph to construct a set of alignment edges; determining a first difference edge from the set of alignment edges; performing multi-dimensional auditing on the first difference edge to determine a candidate difference edge; and determining an abnormal difference edge from the candidate difference edge. In the video auditing, the specified data in multiple dimensions is comprehensively considered and multi-dimensional auditing is performed, thereby improving the auditing effect.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a video review method, apparatus, electronic device, and storage medium. Background Technology

[0002] Current content moderation for videos typically employs methods such as frame extraction, OCR, and sensitive word detection. These methods can identify explicit violations, sensitive words, or inappropriate dialogue. However, these existing methods do not take into account factors such as plot execution, continuous camera movements, and occlusion, resulting in a somewhat one-sided and limited effectiveness of the moderation process. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a video review method, apparatus, electronic device, and storage medium to solve the problems of limited review effectiveness in existing review methods.

[0004] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0005] The first aspect of this invention discloses a video review method, the method comprising:

[0006] Retrieve the video to be reviewed and its associated specified data;

[0007] Based on the video to be reviewed and the specified data, obtain the set of normative constraints, the set of actual observations, and the set of evidence nodes;

[0008] The set of expected constraints, the set of actual observations, and the set of evidence nodes are written into the audit graph to construct the set of aligned edges;

[0009] Determine the first difference edge from the set of aligned edges;

[0010] The first difference edge is reviewed from multiple dimensions to determine candidate difference edges;

[0011] Anomaly edges are identified from the candidate difference edges, and the anomaly edges represent anomalies present in the video to be reviewed.

[0012] Preferably, based on the video to be reviewed and the specified data, a set of ought-to-be constraints, a set of actual observations, and a set of evidence nodes are obtained, including:

[0013] Compile the specified data into a structured set of normative constraints;

[0014] The video to be reviewed is analyzed to obtain the overall observation state, the character-scene occupancy field, and the contact-occlusion state sequence;

[0015] Based on the observed states, the role-scene occupancy field, and the contact-occlusion state sequence, a set of actual observations and a set of evidence nodes are formed.

[0016] Preferably, determining the first difference edge from the set of aligned edges includes:

[0017] Identify lens status;

[0018] Select the difference edge type combination based on the lens status;

[0019] An edge belonging to the combination of differential edge types is instantiated from the set of aligned edges to form the first differential edge that needs to be reviewed.

[0020] Preferably, the first difference edge is subjected to multi-dimensional review to determine candidate difference edges, including:

[0021] The first differential edge is subject to multi-dimensional review;

[0022] The first difference edge that fails the review in any dimension is identified as a candidate difference edge.

[0023] Preferably, determining anomalous difference edges from the candidate difference edges includes:

[0024] Calculate the anomaly detection index for the candidate difference edge;

[0025] Candidate difference edges that satisfy the preset gating conditions are determined to be abnormal difference edges.

[0026] Preferably, after determining the abnormal difference edges from the candidate difference edges, the method further includes:

[0027] A rework contract is generated based on the abnormal differential edge and the evidence node associated with the abnormal differential edge;

[0028] The video to be reviewed shall be repaired in accordance with the repair contract.

[0029] A second aspect of this invention discloses a video review device, the device comprising:

[0030] The first acquisition unit is used to acquire the video to be reviewed and its associated specified data;

[0031] The second acquisition unit is used to obtain the set of normative constraints, the set of actual observations, and the set of evidence nodes based on the video to be reviewed and the specified data.

[0032] A construction unit is used to write the set of normative constraints, the set of actual observations, and the set of evidence nodes into the audit graph to construct an alignment edge set;

[0033] The first determining unit is used to determine the first differential edge from the set of aligned edges;

[0034] The review unit is used to perform multi-dimensional review on the first difference edge to determine the candidate difference edge;

[0035] The second determining unit is used to determine abnormal differential edges from the candidate differential edges, wherein the abnormal differential edges represent anomalies present in the video to be reviewed.

[0036] Preferably, the second acquisition unit is specifically used for:

[0037] Compile the specified data into a structured set of normative constraints;

[0038] The video to be reviewed is analyzed to obtain the overall observation state, the character-scene occupancy field, and the contact-occlusion state sequence;

[0039] Based on the observed states, the role-scene occupancy field, and the contact-occlusion state sequence, a set of actual observations and a set of evidence nodes are formed.

[0040] A third aspect of the present invention discloses an electronic device, comprising: a processor and a memory, the processor and the memory being connected via a bus; wherein, the processor is used to call and execute a program stored in the memory; the memory is used to store the program, the program being used to implement the video review method disclosed in the first aspect of the present invention.

[0041] A fourth aspect of the present invention discloses a storage medium storing computer-executable instructions for executing the video review method disclosed in the first aspect of the present invention.

[0042] Based on the above embodiments of the present invention, a video review method, apparatus, electronic device, and storage medium are provided. The method comprises: acquiring the video to be reviewed and its associated specified data; obtaining a set of expected constraints, a set of actual observations, and a set of evidence nodes based on the video to be reviewed and the specified data; writing the set of expected constraints, the set of actual observations, and the set of evidence nodes into a review graph to construct an alignment edge set; determining a first difference edge from the alignment edge set; performing multi-dimensional review on the first difference edge to determine candidate difference edges; and determining abnormal difference edges from the candidate difference edges. This method comprehensively considers multi-dimensional specified data and performs multi-dimensional review in video review, thereby improving the review effect. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0044] Figure 1 A flowchart of a video review method provided in an embodiment of the present invention;

[0045] Figure 2 This is an example structural diagram of the CTAG audit diagram provided in an embodiment of the present invention;

[0046] Figure 3 Example diagram of target consistency detection provided in embodiments of the present invention;

[0047] Figure 4 Example diagram of space occupancy detection provided in embodiments of the present invention;

[0048] Figure 5 This is an example diagram of the penetration detection provided in an embodiment of the present invention;

[0049] Figure 6 Example diagram of the audit evidence chain provided in the embodiments of the present invention;

[0050] Figure 7 This is an example diagram of a closed-loop repair contract provided in an embodiment of the present invention;

[0051] Figure 8 This is a structural block diagram of a video review device provided in an embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0054] Current content moderation for videos typically employs methods such as video frame extraction, OCR, ASR, facial recognition, object detection, sensitive word retrieval, image classification, and text classification. These methods can identify explicit violations, sensitive words, or inappropriate dialogue. Existing video generation evaluation schemes usually focus on subject consistency, text-video consistency, image quality, motion quality, physical plausibility, or overall score. However, when it comes to producing AI-generated video clips (such as AI-generated short dramas), the following shortcomings still exist:

[0055] First, the review system is incomplete. Compliance review, image quality scoring, subject consistency and motion quality are often handled separately, making it difficult to cover issues such as plot execution, cross-shot continuity, spatial positioning, contact occupancy, clipping, prop connection, semantic consistency between dialogue and screen, and the rationality of camera language.

[0056] Second, the review granularity is at the frame level, segment level, or whole-film level, lacking a structure that aligns the expected constraints with the actual observations by object, anchor point, and attribute;

[0057] Third, existing target consistency checks often use uniform similarity or uniform position thresholds to judge all shots, failing to distinguish between static shots from fixed positions and continuous motion shots. In static or fixed shots, the targets, scene anchors, and relative positions of consecutive frames should be basically consistent; in continuous motion shots, pixel position changes are normal phenomena, and their reasonableness should be judged based on motion compensation, spatial continuity, trajectory smoothness, and depth occlusion relationships. If the same rules are used for both types of shots, normal motion is easily misjudged as drift, and target replacement or spatial drift that gradually occurs during the motion may be missed.

[0058] Fourth, the review and rework processes are not linked. Even if the review model gives the correct conclusion, the output often remains at the level of labels or descriptions such as "wrong, low score, suspected inconsistency", and cannot be automatically transformed into appropriate rework methods such as "where to repair, what to lock, what to change, and how to accept".

[0059] Fifth, single-point review agents or single quality inspection modules usually make judgments based only on local frames, local objects, single modalities, or local confidence levels. They lack unified arbitration of plot status, shot semantics, cross-frame continuity, and multimodal evidence chains. This can easily lead to misjudging normal transitions, character movements, camera movements, short-term occlusion, or reasonable contact as quality defects, and directly passing these misjudgments to the rework process, resulting in excessive re-creation or repairing shots that were originally correct.

[0060] To address this, embodiments of the present invention propose a video review method, apparatus, electronic device, and storage medium. The method involves acquiring the video to be reviewed and its associated specified data; obtaining a set of expected constraints, a set of actual observations, and a set of evidence nodes based on the video and the specified data; writing these sets into a review graph to construct an alignment edge set; determining a first difference edge from the alignment edge set; performing multi-dimensional review on the first difference edge to determine candidate difference edges; and identifying abnormal difference edges from the candidate difference edges. By comprehensively considering multi-dimensional specified data and performing multi-dimensional review in video review, the review effectiveness is improved.

[0061] See Figure 1 The flowchart of a video review method provided by an embodiment of the present invention is shown. The method includes:

[0062] Step S101: Obtain the video to be reviewed and its associated specified data.

[0063] In the specific implementation step S101, the video to be reviewed (e.g., a completed AI short drama) and the specified data associated with the video to be reviewed are obtained. This specified data is used to generate the normative constraints for reviewing the video to be reviewed.

[0064] The specified data associated with the video to be reviewed includes, but is not limited to: script (or screenplay), storyboard, prompts, character settings, scene settings, and reference images.

[0065] In some embodiments, a storyboard-shot-time period hierarchical index is established, and shot state L is identified, which includes at least fixed camera position (or static shot) and continuous moving shot.

[0066] Step S102: Based on the video to be reviewed and the specified data, obtain the set of normative constraints, the set of actual observations, and the set of evidence nodes.

[0067] It should be noted that the normative constraint set is used to provide the review benchmark, the actual observation set is used to provide the actual observation status of the video to be reviewed, and the evidence node set is used to provide traceable evidence information for the difference edge.

[0068] In the specific implementation step S102, the specified data is compiled into a structured set of normative constraints R.

[0069] For example, the script, storyboard, cues, character settings, scene settings, and reference diagrams are compiled into a structured set of normative constraints R through NCS (Narrative Constraint State).

[0070] The normative constraint set R includes at least the following fields: character identity, scene anchor point, key props, action sequence, dialogue semantics, spatial occupancy, and contact relationship.

[0071] For example, the normative constraint set R should include at least: character identity, scene / target, props, space occupancy, contact occlusion, action continuity, dialogue / visual semantics, and compliance constraints.

[0072] It should be noted that "major targets" refer to the main objects in a video scene that serve as the visual focus or key elements of the plot, such as a wooden table in a coffee shop scene, a medicine box in a pharmacy scene, or specific props held by characters. "Major targets" and "scenes" (referring to the background environment, such as a coffee shop or hospital ward) together constitute the consistency review objects for scenes / major targets.

[0073] Understandably, platform hard rules, compliance rules, and manual confirmation rules can also be incorporated into the normative constraint set R.

[0074] The video to be reviewed is parsed to obtain the actual observation set M and the evidence node set N. Specifically, the video is parsed to obtain supporting observation states such as VOS (Video Observation State), RSOF (Role-Scene Occupancy Field), and COSS (Contact-Occlusion State Sequence). Based on the actual observation state, the role-scene occupancy field, and the contact-occlusion state sequence, the actual observation set M and the evidence node set N are formed.

[0075] It should be noted that VOS (Video Observation Status) is responsible for extracting keyframes, shot segmentation, target areas, character / prop detection results, ASR text, OCR text, camera movement, confidence level, and other basic observations from the video to be reviewed, forming the main source of the actual observation set M.

[0076] RSOF (Role-Site Placeholder Field) is based on VOS and uses scene anchor points as references to express the relative position, orientation, depth level, occlusion relationship and trajectory continuity of characters, large objects and props. It is used to support scene consistency, spatial placeholder continuity and cross-axis / swapping anomaly review.

[0077] COSS (Contact-Obscuration State Sequence) is used to express continuous states such as approach, contact, holding, supporting, obscuring, releasing, and impenetrable, and is used to support the review of contact, obscuring, clipping, and depth conflict between hands / human bodies / props.

[0078] The outputs of VOS, RSOF, and COSS are mapped to the actual observation set M and the evidence node set N.

[0079] It should be further noted that the actual observation set M includes at least: storyboard / shot / time index, object identifier, identity and appearance features, scene anchor point, relative position and orientation, depth and occlusion relationship, trajectory / velocity, contact state, action stage, ASR / OCR semantic results and corresponding confidence.

[0080] The evidence node set N is a traceable evidence carrier of the actual observation set M. The evidence node set N stores at least: keyframes, time periods, region boxes, masks, key point trajectories, depth sorting, original ASR / OCR fragments, model sources, time locations, quality information, confidence levels, and relevant script terms.

[0081] An M item can reference one or more N nodes.

[0082] Step S103: Write the set of normative constraints, the set of actual observations, and the set of evidence nodes into the audit graph to construct the set of aligned edges.

[0083] It should be noted that the audit graph is CTAG (Constraint Traceback Audit Graph), which is the core audit and rework data structure. This audit graph can be represented by formula (1).

[0084] CTAG=(R,M,N,E)(1;

[0085] In formula (1), R is the set of normative constraints, M is the set of actual observations, N is the set of evidence nodes, and E is the set of difference edges. Each difference edge e in the set of difference edges E connects one or more normative constraints with the corresponding actual observations and binds evidence nodes and rework targets.

[0086] Each differential edge e includes at least the following fields: scope, anchor, property, lens_state, edge_group, expected, observed, diff, T, gate, evidence, and repair_target.

[0087] Specifically, `scope` represents the scope of application, including episodes, storyboards, shots, time periods, and regions; `object` represents objects such as characters, scenes, props, hands, faces, or dialogue; `anchor` represents scene anchor points, adjacent shots, character body parts, or objects in contact; `property` represents the attribute being reviewed, such as identity, appearance, location, orientation, depth, occlusion, contact, action, or semantics; `lens_state` represents the shot state `L`; `edge_group` represents the difference edge combination `B` enabled in the current state; `expected` represents the expected value, and `observed` represents the actual value; `diff` represents the difference function; `T` represents the threshold; `gate` represents the abnormal causal gating condition; `evidence` represents the evidence node index; and `repair_target` represents the repair target.

[0088] In the specific implementation step S103, the set of normative constraints R, the set of actual observations M, and the set of evidence nodes N are written into the audit graph CTAG. A comparable set of alignment edges is established based on the alignment key alignment_key. The set of alignment edges contains multiple differential edges.

[0089] Furthermore, when constructing the CTAG (Consideration Chart), an alignment key (alignment_key) is first generated for the normative constraint set R and the actual observation set M. This alignment key consists of the storyboard number, shot number, object identifier, scene anchor point, and attribute type. R and M sets with the same or mappable alignment_key are organized onto the same difference edge, thus forming an alignment edge set.

[0090] Step S104: Determine the first difference edge from the set of aligned edges.

[0091] In the specific implementation of step S104, a storyboard-shot-time period hierarchical index is established, and shot state L is identified. Shot state L includes at least fixed camera position (or static shot) and continuous moving shot.

[0092] The differential edge type combination B_L is selected based on the shot status. It should be noted that the differential edge type combination B_L is an edge template or activation rule used during the construction and calculation of the CTAG (Conceptual Graph). The uses of the differential edge type combination B_L are: 1. To determine which comparison components are enabled for fixed or moving shots; 2. To determine the weights, thresholds, and filtering conditions of each component in the stateful differential intensity D_L(e); 3. To determine which differential edges enter the candidate differential edge selection, and which differential edges enter causal gating and evidence convergence.

[0093] Instantiate an edge (i.e., a differential edge) belonging to the differential edge type combination B_L from the aligned edge set to form the first differential edge that needs to be reviewed. That is, instantiate or activate a differential edge belonging to the differential edge type combination B_L to form the first differential edge that needs to be reviewed for the current shot.

[0094] Step S105: Perform multi-dimensional review on the first difference edge to determine candidate difference edges.

[0095] In the specific implementation step S105, the first differential edge is subject to multi-dimensional review. The multi-dimensional review includes any of the following: role identity consistency review, scene or large target consistency review, spatial occupancy continuity review, contact occlusion and clipping review, action and timing continuity review, and dialogue and screen semantic consistency review.

[0096] The first difference edge that fails the review in any dimension is identified as a candidate difference edge.

[0097] The following explains each aspect of the multi-dimensional audit:

[0098] R1, Character Identity Consistency Verification: Used to detect identity drift of the same character in terms of face, clothing, hairstyle or voice.

[0099] R2, Scene or Major Object Consistency Check: Used to detect the replacement of background, main objects or key objects in the same plot space, and select different difference edge combinations according to fixed shots or moving shots.

[0100] R3, Spatial Occupation Continuity Check: Used to detect sudden changes in the position of characters relative to the scene anchor point in static scenes, crossing the axis or swapping positions, as well as discontinuities in trajectory, speed and depth occlusion relationships in dynamic scenes.

[0101] R4. Contact Occlusion and Clipping Inspection: Used to detect fingers passing through the cup wall, bodies passing through tables and chairs, props being incorrectly occluded, or depth sorting conflicts.

[0102] R5. Action and Timing Continuity Audit: Used to detect discontinuities in action direction, speed, posture, or causal sequence.

[0103] R6. Verbal and visual semantic consistency audit: Used to detect conflicts between dialogue, subtitles, visual objects, and plot states.

[0104] The multi-dimensional audits given in R1 to R6 above are not isolated decision-makers, but candidate differential edge generators in the audit graph CTAG. Candidate differential edges share evidence nodes, gating logic, and rework targets. Only after passing the abnormal causal gating can a candidate differential edge be determined as an abnormal differential edge, and only then will the abnormal differential edge enter the rework plan.

[0105] Understandably, when performing the aforementioned multi-dimensional review on the first difference edge, R1 to R6 respectively read and review the first difference edge related to their respective dimensions. R1 outputs the consistency result of the character's identity; R2 outputs the consistency result of the scene and the main image; R3 outputs the spatial occupancy and trajectory continuity results; R4 outputs the contact, occlusion, and clipping results; R5 outputs the action and temporal continuity results; and R6 outputs the semantic consistency result of the dialogue, subtitles, and visuals. The review results for each dimension should at least include the review scope, object, anomaly type, expected value, actual value, difference value, initial confidence level, and evidence node.

[0106] When the initial difference of the first difference edge calculated in a certain dimension reaches the candidate threshold, the first difference edge is determined as a candidate difference edge, and the corresponding candidate anomaly is written on the candidate difference edge.

[0107] Step S106: Identify abnormal differential edges from the candidate differential edges. Abnormal differential edges represent anomalies in the video to be reviewed.

[0108] In the specific implementation of step S106, abnormal causal gating is performed on the candidate difference edges to identify abnormal difference edges.

[0109] It should be noted that the anomaly causal gating is used to further determine whether candidate differential edges discovered through multi-dimensional audits are "explainable differentials" or "genuine anomalies." It serves as a unified arbitration layer between quality audits and rework planning. Instead of directly triggering rework based on single-frame deviations, single classification scores, or local conclusions of a particular agent, it calculates anomaly judgment indicators for candidate differential edges (which can also be understood as differential edges enabled by the differential edge type combination B_L), and then determines whether the candidate differential edge is an anomalous differential edge (i.e., whether it is a genuine anomaly).

[0110] In some embodiments, the determination of abnormal difference edges is achieved by: calculating an anomaly determination index for candidate difference edges, wherein the anomaly determination index includes at least the lens state-based difference intensity D. L (e) Interpretive strength G(e) and evidentiary reliability Q(e). Candidate difference edges that satisfy the preset gating conditions are identified as anomalous difference edges.

[0111] The following sections provide detailed explanations of the anomaly detection indicators and preset gating conditions:

[0112] Lens-based differential intensity D L (e) The distance is obtained by normalizing the normative constraints and actual observations under the current shot state L, with a value range of 0 to 1; for example, the identity difference can be obtained by weighting the embedding distance of the face, clothing and voice, the spatial difference can be obtained by the position, orientation, depth and trajectory offset of the character relative to the scene anchor point, and the semantic difference can be obtained by the consistency distance of the dialogue, subtitles and screen objects.

[0113] Furthermore, the lens-state differential intensity D L (e) is calculated using formula (2).

[0114] (2);

[0115] In formula (2), k is a comparison component in the differential edge type combination B_L, such as identity, appearance, scene anchor, position, trajectory or depth occlusion; Let be the normalized difference value of the difference edge e (which can represent the candidate difference edge) at the kth term. The larger the value, the more obvious the deviation. denoted as the weight of the k-th item under shot state L, used to reflect the degree of importance different shots place on different review items.

[0116] After obtaining D L After (e), construct the plot state vector S(e) and the camera state L. The plot state vector S(e) includes at least the following states: static dialogue, character movement, camera movement, transition, time jump, dream / memory, montage, inserted shot, short occlusion and contact action.

[0117] Simultaneously extract camera movement interpretations (P) from VOS, RSOF, and COSS. k (e) Plot Permissibility s (e) Explanation of contact occlusion P c (e) and the reliability of evidence Q(e). The explanatory strength G(e) can be calculated using the rule-learning hybrid gating model as shown in Equation (3).

[0118] (3);

[0119] In formula (3), the plot allowance P s (e) Indicates whether the current plot allows this difference to occur; Camera movement interpretation P k (e) Indicates the strength of the explanation for differences caused by push, pull, pan, tilt, zoom, cut, or transition; contact occlusion explanation P c (e) indicates whether the contact, occlusion and depth sequences satisfy physical continuity; evidence reliability Q(e) indicates the reliability of the evidence node; w1 to w5 are weights, b is the bias, which can be configured by expert rules or obtained by training with a small number of labeled samples.

[0120] The higher the explanatory strength G(e) of a candidate difference edge, the more likely the candidate difference edge can be explained by normal narrative or normal cinematic language; the lower the explanatory strength G(e), the less likely the candidate difference edge lacks a reasonable causal explanation.

[0121] Therefore, satisfying the preset gating condition means: DL (e)>T d And G(e) <T g And Q(e)>T q .

[0122] Among them, T d T is the difference threshold. g To explain the threshold, T q This is the evidence threshold.

[0123] For a given candidate difference edge, when the D of the candidate difference edge... L (e)>T d And G(e) <T g And Q(e)>T q If the difference is large, the candidate difference edge is not reasonably explained and the evidence is reliable. Therefore, the candidate difference edge is identified as a true anomaly through gating, meaning it is an anomalous difference edge. Otherwise, A(e) = 0. This indicates that the difference is small or can be explained by normal plot, camera movement, transitions, occlusion, etc., and is not treated as an anomaly; therefore, the candidate difference edge is not an anomalous difference edge.

[0124] In other words, the "abnormal judgment index meets the preset gating condition" can be expressed by formula (4).

[0125] A(e) = I[D L (e)>T d ^ G(e) <T g ^ Q(e)>T q (4);

[0126] In formula (4), A(e) is the anomaly marker, I[·] is the indicator function, and T d T is the difference threshold. g To explain the threshold, T q The threshold for evidence is D. L (e) represents the intensity of the lens state differentiation. G(e) represents the intensity of interpretation. Q(e) represents the reliability of the evidence.

[0127] As can be seen from the above content on judging abnormal differential edges, the CTAG review image does not directly judge all differential edges as wrong. Instead, it first judges whether the differential edge violates the required differential edge combination in the current shot state, and whether it can be reasonably explained by the plot state, camera movement, transition or contact occlusion state.

[0128] Understandably, in the process of determining whether a candidate difference edge is an abnormal difference edge, if "D" is not satisfied... L (e)>T d And G(e) <T g And Q(e)>Tq This preset gating condition can either set the anomaly flag A(e) of the candidate difference edge to 0, or delay the verification of the candidate difference edge.

[0129] Among them, delayed verification is characterized by: the difference looks suspicious, but the current evidence is insufficient, multiple modal conclusions conflict, or the value is close to the threshold, so A(e) is temporarily uncertain and awaits verification in subsequent frames, adjacent shots, or manual verification.

[0130] For example, if a candidate anomaly of a candidate difference edge is rejected due to reasons such as plot, camera movement, occlusion, camera state, or insufficient evidence, then a delayed review can be performed.

[0131] For example, if a character's face cannot be recognized temporarily when it is obscured, it cannot be directly judged as an abnormal identity, nor can it be immediately judged as normal. The screen should be reviewed after the obscuration is lifted.

[0132] It should be further noted that the abnormal causal gating dynamically adjusts the threshold and weight under different plot states and camera states. If camera state L is a fixed camera position or static shot, then B is activated. F ={Target identity edge, appearance edge, scene anchor point edge, and position edge of previous and next frames}, with stricter thresholds applied to target replacement, appearance abrupt changes, anchor point drift, and unexplained position jumps. Among these, B F For fixed camera positions or static shots, the difference parameters are combined, primarily comparing the target's identity, appearance, scene anchor points, and relative positions between consecutive frames. Since the camera remains largely stationary, significant changes in these elements are more likely to be considered anomalies.

[0133] If the camera state L is a continuous motion shot, then enable B. M ={Target Identity Edge, Motion Compensation Edge, Trajectory Continuity Edge, Spatial Occlusion Edge, Depth Occlusion Edge}, does not require the pixel position to remain completely unchanged, but requires that the trajectory, velocity, orientation, relative anchor point, and occlusion changes after camera movement compensation be continuous and interpretable. Among them, B M The differential combination used for continuous motion shots mainly compares identity continuity, trajectory after motion compensation, speed, relative scene anchor point, spatial occupancy, depth, and occlusion continuity. Pixel position changes in motion shots are normal and anomalies cannot be directly judged based on the positions of preceding and following frames.

[0134] If the plot state vector S(e) represents a transition, time jump, dream, or montage, some spatial continuity constraints are downgraded or delayed in their determination. However, character identities, key props, and plot-essential objects retain minimum constraints to avoid complete exemption by the narrative tag. S(e) determines whether the current discrepancy can be reasonably explained by plot development, and adjusts the strength of the explanation, review threshold, or delay of the review process accordingly.

[0135] Therefore, the difference of the same target may be identified as abnormal in a fixed shot, but may be gated and interpreted as reasonable motion in a continuous moving shot; conversely, if the identity edge is continuous and stable in a moving shot, but the trajectory continuous edge, spatial occupancy edge and depth occlusion edge are all abnormal, the system can determine that it is target drift or spatial drift during the motion process.

[0136] For candidate anomalies of contact occlusion and clipping, the contact occlusion explanation P is calculated using the COSS state sequence. c (e) Determined not solely by target box overlap. When the "approach-contact-hold-release" sequence is complete, the mask overlap change matches the hand keypoint trajectory, and depth sorting is simultaneously valid, P... c (e) Increase, differential edges are interpreted as normal occlusion or reasonable contact; when keypoints continuously cross object boundaries, masks continuously overlap abnormally, depth sorting conflicts occur, and the action intent cannot be interpreted, P c (e) Reduce and identify the differential edge as a clipping or contact relationship breakage anomaly. If multiple low-confidence edges of the same object or the same scene anchor point appear consecutively, a cross-shot cumulative anomaly score H(o) can also be formed. The calculation formula of H(o) is shown in formula (5).

[0137] (5);

[0138] In formula (5), E o To represent a set of candidate difference edges related to the same object o, object o can be a character, prop, scene anchor, or other target. To avoid the accumulation of unrelated differences, E o The difference edges in the data should also satisfy the following conditions: they are located in adjacent time periods or shots, and the anomalies are of the same type or related to each other.

[0139] For example: o represents "blue medicine box", E o It may include the appearance difference edge, identity difference edge, and positional continuity difference edge of the pillbox in multiple consecutive shots.

[0140] In practical applications, H(o) is used to detect drift problems that are not obvious in a single frame but worsen continuously. When the same object repeatedly exhibits weak differences with the same direction in consecutive frames or adjacent shots, even if a single difference edge has not yet met the anomaly confirmation condition, the accumulated H(o) may exceed the preset cumulative anomaly threshold T. h At this point, the object is identified as having progressive identity drift, appearance drift, or spatial drift anomalies.

[0141] When the same object o has multiple candidate difference edges of the same type and with the same direction in consecutive frames or adjacent shots, and H(o) exceeds the cumulative anomaly threshold T h If H(o) is positive, then the object is confirmed to have a cumulative anomaly; otherwise, the anomaly is not confirmed based on H(o).

[0142] In some embodiments, after identifying abnormal differential edges from candidate differential edges, only the abnormal differential edges confirmed by abnormal causal gating in the audit graph CTAG and their associated evidence nodes are read. Based on the abnormal differential edges and the evidence nodes associated with the abnormal differential edges, a repair contract (RC) is generated to avoid passing the reclaimed candidate misjudgments to regeneration or manual post-processing.

[0143] The rework contract must include at least the fields loc, lock, fix, target, and accept.

[0144] `loc` records storyboards, shots, time periods, and regions; `lock` records character identities, prop appearances, scenes, compositions, camera movements, and reference frames that must remain unchanged during rework; `fix` records the scope of local redraws, local rebirths, shot regenerations, voice-over re-dubbing, or manual post-production; `target` records target characters, target scene anchor points, spatial occupant targets, contact / occlusion targets, and semantic targets; `accept` records acceptance predicates, such as identity consistency, scene anchor point consistency, no impossible penetration, dialogue / visual consistency, and risk score below a threshold.

[0145] The videos awaiting review will be repaired in accordance with the repair contract. Specifically, this involves using modules such as partial redrawing, shot regeneration, dubbing re-dubbing, or manual post-production to repair the videos according to the repair contract.

[0146] Re-enter the rework results into the audit diagram CTAG, and determine whether to pass, continue iteration, or transfer to manual review based on the accept predicate.

[0147] The above is a detailed description of the present invention. The process involves: acquiring the video to be reviewed and its associated specified data; obtaining a set of expected constraints, a set of actual observations, and a set of evidence nodes based on the video and the specified data; writing the set of expected constraints, the set of actual observations, and the set of evidence nodes into the review graph to construct an alignment edge set; determining a first difference edge from the alignment edge set; performing multi-dimensional review on the first difference edge to determine candidate difference edges; and determining abnormal difference edges from the candidate difference edges. This approach comprehensively considers multi-dimensional specified data and performs multi-dimensional review in video review, thereby improving the review effect.

[0148] Furthermore, to better understand the audit diagram CTAG, multi-dimensional audit, and overall process mentioned in the present invention, some examples will be provided below.

[0149] like Figure 2The example diagram of the CTAG auditing structure shows that the left side is the normative constraint (NCS) generated from the storyboard, prompts, and character settings, while the right side is the actual constraint (VOS), response time (RSOF), and control order (COSS) obtained from the video to be audited (e.g., a short drama). The CTAG differential auditing tool aligns the normative constraints and actual observations according to storyboard, shot, object, anchor point, and attribute to generate differential edges. When the differential exceeds a threshold and the anomalous causal gating cannot provide a credible explanation, it outputs anomaly differential edges, evidence nodes, and risk results. Figure 2 The advantage of CTAG is that it does not only give a quality score, but also preserves a structured chain of information such as "what constraints were violated, what the actual deviation was, where the evidence is, and how to rework it".

[0150] In other words, CTAG differs from ordinary knowledge graphs or simple scoring tables. Ordinary graphs primarily describe object relationships, and scoring tables mainly output overall scores; CTAG places "how it should be, how it actually is, how big the difference is, why it's abnormal, where the evidence is, and how to revisit" on the same differential edge and its associated evidence nodes. Therefore, CTAG is both a review and judgment structure and a revisit planning structure. It can uniformly express contact anomalies within the same shot, large-scale drifts in adjacent shots, spatial occupancy jumps in adjacent scenes, and semantic inconsistencies between dialogue and visuals as calculable, traceable, and verifiable anomaly subgraphs.

[0151] Regarding the above multi-dimensional review, such as Figure 3 As shown, Figure 3 An example diagram of target consistency testing is shown. Figure 3 This is used to verify the consistency of major targets and scenes within the same or adjacent storyboards. For fixed camera positions or static shots, the system obtains the scene anchor points, main targets, and prop constraints that should be consistent from the NCS (Non-Consistent Shot System), and then extracts keyframes, appearance features, target regions, and semantic tags from the VOS (Voice of Targets) to form the target's identity edge, appearance edge, scene anchor point edge, and position edge of preceding and following frames in the CTAG (Consistent Target Aggregate). When the subject drifts from A to B without a transition, inserted shot, or plot explanation, the system outputs an abnormal edge and binds it to the keyframes of the preceding and following shots as evidence.

[0152] like Figure 4 As shown, Figure 4 An example diagram of space occupancy detection is shown. Figure 4 This is used to illustrate the continuity review of character spatial positioning and to demonstrate the different review rigor between fixed shots and continuously moving shots. In static or fixed camera positions, the character's position, orientation, depth level, and occlusion relationship relative to scene anchor points should remain stable. In continuously moving shots, the pixel position of the target or character can change, but the trajectory, speed, orientation, relative anchor point, and occlusion changes after camera movement compensation should be continuous. CTAG records these relationships using RSOF and uses anomaly causal gating to distinguish between normal movement, camera movement, cross-axis interpretation, and unexplained jumps.

[0153] like Figure 5 As shown, Figure 5 An example diagram of mold detection is shown. Figure 5 This is used to illustrate the difference between normal contact sequences and clipping anomalies in a handheld cup scene. Normally, the hand and cup go through states such as approach, contact, holding, and release; in abnormal situations, fingers pass through the cup wall and enter the cup's interior, resulting in continuous mask overlap, depth sorting conflicts, and keypoint out-of-bounds errors. Based on this, a COSS is generated, and contact occlusion differential edges are formed in the CTAG, providing local rework areas and contact targets for the RC (Revised Constructor).

[0154] An example diagram of the evidence chain for this scheme is shown below. Figure 6 As shown, Figure 6 Used to demonstrate the verifiability of audit conclusions. Abnormal areas in problematic footage are linked to keyframes, region boxes, masks, depth conflicts, keypoint trajectories, COSS states, difference values, and script constraints, forming evidence nodes. The evidence chain is not a single model label, but a structured record of "abnormal difference edge - evidence node - rework target," which can be directly used for manual review, risk grading, or rework contract generation.

[0155] Overall, the closed-loop process of "repair contract - re-review" is as follows: Figure 7 As shown, Figure 7 This describes the closed-loop process from review to rework. Anomaly differential edges are first aggregated into an evidence package, and the rework plan generates a Rework Contract (RC) based on this package. The RC includes locked assets, rework time period, rework area, spatial occupancy targets, contact / occlusion targets, semantic targets, and acceptance conditions, driving local redrawing, shot regeneration, or manual post-production. The regenerated result re-enters the CTAG for review; if it fails, it iterates further; if it passes, the final result is output.

[0156] After applying this solution, the application scenarios will be as follows:

[0157] 1. Receive a video (e.g., a short drama) and its storyboard for review. The storyboard description is "The female lead shows the male lead a blue medicine box in the same coffee shop." Extract scene anchors, characters, props, actions, and dialogue constraints from the storyboard to form the R; then extract shots, keyframes, target areas, character identities, prop appearances, ASR dialogue, and OCR subtitles from the finished video to form the M. Generate alignment_keys by storyboard number, object, anchor, and attribute, and generate scene consistency edges, target consistency edges, spatial placement edges, contact occlusion edges, and semantic consistency edges in CTAG.

[0158] 2. If in the same shot, the previous shot of a fixed camera position shows a wooden table in a coffee shop and a blue medicine box, and the next frame or shot suddenly changes to a white wall or white medicine bottle in a hospital, and the script does not contain transition, flashback, or location switching information, then CTAG will enable B_F to perform strong constraint comparison on the target identity edge, appearance edge, scene anchor point edge, and position edge of the previous and next frames; when the difference value exceeds the threshold and the gating interpretation is insufficient, the abnormal difference edge will be output and bound to the previous and next keyframes as evidence.

[0159] If the camera movement is a push, pull, pan, tilt, or character tracking shot, displacement of the target in pixel coordinates is not directly considered abnormal. Using B_M (Browser-Modulated), camera movement compensation is performed first, then the target's identity is compared to its continuity, trajectory smoothness, continuity relative to scene anchor points, and the interpretability of depth and occlusion relationships. If the identity edge is stable and the trajectory continuous, it is considered reasonable movement. If there is unexplained target replacement, trajectory breakage, or relative anchor point jumps, the target's drifting abnormal edge is output.

[0160] 3. If the script requires a character to pick up a cup, the normal sequence should be approach, touch, hold, and release. If, in the final cut, a finger keypoint is found to pass through the cup wall, and the cup mask and finger mask continuously and unreasonably overlap, and the depth sort cannot explain this overlap, then CTAG will mark this contact occlusion edge as an abnormal differential edge. The rework contract will lock the character's identity, the cup's appearance, and composition, requiring only the rework of the area where the finger meets the cup wall, with "no impossible penetration, continuous contact state, and consistent appearance" as acceptance criteria.

[0161] 4. Read the abnormal differential edges and evidence nodes, generate a rework contract (RC), and then hand it over to the local redraw, shot regeneration, or manual post-production module. The rework result is re-entered into CTAG for review; if the abnormal differential edges disappear and the acceptance predicate passes, a qualified final cut is output; if abnormalities still exist, new evidence nodes are added to CTAG and the next round of RC is generated.

[0162] In summary, this solution transforms AI video review from a single score into interpretable differential edges, allowing anomalies to be traced back to specific constraints, shots, regions, and evidence. CTAG unifies plot constraints, video observation, evidence nodes, and rework targets, avoiding semantic breaks between the generation, review, and rework systems. The unified anomaly causal gating incorporates candidate anomalies from multiple single-point agents into the same causal arbitration mechanism, combining plot status, camera movement status, contact / occlusion status, and evidence reliability to recover misjudgments and reduce false alarms caused by normal transitions, character movements, and occlusion. RC only accepts anomaly differential edges confirmed through gating, giving rework agents clear boundaries, enabling them to lock assets that should not be modified and only rework problematic areas, reducing excessive regeneration, missed repairs, and human communication costs. It does not rely on the internal parameters of a specific video generation model and can be used as a black-box review layer to connect to various AIGC video production platforms.

[0163] Corresponding to the video review method provided in the above embodiments of the present invention, see also... Figure 8 The present invention also provides a structural block diagram of a video review device, including: a first acquisition unit 801, a second acquisition unit 802, a construction unit 803, a first determination unit 804, a review unit 805, and a second determination unit 806.

[0164] The first acquisition unit 801 is used to acquire the video to be reviewed and its associated specified data. The specified data is used to generate the normative constraints for reviewing the video to be reviewed.

[0165] The second acquisition unit 802 is used to obtain a set of normative constraints, a set of actual observations, and a set of evidence nodes based on the video to be reviewed and specified data. The set of normative constraints is used to provide the review benchmark, the set of actual observations is used to provide the actual observation status of the video to be reviewed, and the set of evidence nodes is used to provide traceable evidence information for the differential edge.

[0166] In specific implementation, the second acquisition unit 802 is specifically used to: compile the specified data into a structured set of normative constraints; parse the video to be reviewed to obtain the overall observation state, the role-scene occupancy field and the contact-occlusion state sequence; and form the actual observation set and the evidence node set based on the overall observation state, the role-scene occupancy field and the contact-occlusion state sequence.

[0167] Construction unit 803 is used to write the set of normative constraints, the set of actual observations, and the set of evidence nodes into the audit graph to construct the set of aligned edges.

[0168] The first determining unit 804 is used to determine the first differential edge from the set of aligned edges.

[0169] In specific implementation, the first determining unit 804 is specifically used for: identifying the lens state; selecting the differential edge type combination according to the lens state; and instantiating the edge belonging to the differential edge type combination from the aligned edge set to form the first differential edge that needs to be reviewed.

[0170] The review unit 805 is used to conduct multi-dimensional review of the first difference edge in order to determine the candidate difference edge.

[0171] In specific implementation, the review unit 805 is used to: perform multi-dimensional review on the first difference edge; and determine the first difference edge that fails any dimension review as a candidate difference edge.

[0172] The multi-dimensional review includes any of the following: role identity consistency review, scene or major title consistency review, spatial occupancy continuity review, contact occlusion and clipping review, action and timing continuity review, and dialogue and screen semantic consistency review.

[0173] The second determining unit 806 is used to determine abnormal differential edges from the candidate differential edges, wherein the abnormal differential edges represent the abnormalities present in the video to be reviewed.

[0174] In specific implementation, the second determining unit 806 is specifically used to: calculate the anomaly judgment index of the candidate difference edge; and determine the candidate difference edge whose anomaly judgment index meets the preset gating condition as an abnormal difference edge.

[0175] Preferred, combined Figure 8 The device, as shown, also includes:

[0176] The rework unit is used to generate a rework contract based on the abnormal differential edge and the evidence node associated with the abnormal differential edge; and to rework the video to be reviewed according to the rework contract.

[0177] Preferably, embodiments of the present invention also provide an electronic device, including: a processor and a memory, the processor and the memory being connected via a bus; wherein, the processor is used to call and execute a program stored in the memory; the memory is used to store the program, the program being used to implement the video review method provided in the above method embodiments.

[0178] Preferably, the present invention also provides a storage medium storing computer-executable instructions for executing the video review method provided in the above-described method embodiments.

[0179] In summary, embodiments of the present invention provide a video review method, apparatus, electronic device, and storage medium. The method involves acquiring a video to be reviewed and its associated specified data; obtaining a set of expected constraints, a set of actual observations, and a set of evidence nodes based on the video and the specified data; writing these sets into a review graph to construct an alignment edge set; determining a first difference edge from the alignment edge set; performing multi-dimensional review on the first difference edge to determine candidate difference edges; and identifying abnormal difference edges from the candidate difference edges. By comprehensively considering multi-dimensional specified data and performing multi-dimensional review in video review, the review effectiveness is improved.

[0180] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0181] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0182] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A video review method, characterized in that, The method includes: Retrieve the video to be reviewed and its associated specified data; Based on the video to be reviewed and the specified data, obtain the set of normative constraints, the set of actual observations, and the set of evidence nodes; The set of expected constraints, the set of actual observations, and the set of evidence nodes are written into the audit graph to construct the set of aligned edges; Determine the first difference edge from the set of aligned edges; The first difference edge is reviewed from multiple dimensions to determine candidate difference edges; Anomaly edges are identified from the candidate difference edges, and the anomaly edges represent anomalies present in the video to be reviewed.

2. The method according to claim 1, characterized in that, Based on the video to be reviewed and the specified data, obtain the set of normative constraints, the set of actual observations, and the set of evidence nodes, including: Compile the specified data into a structured set of normative constraints; The video to be reviewed is analyzed to obtain the overall observation state, the character-scene occupancy field, and the contact-occlusion state sequence; Based on the observed states, the role-scene occupancy field, and the contact-occlusion state sequence, a set of actual observations and a set of evidence nodes are formed.

3. The method according to claim 1, characterized in that, Determining the first difference edge from the set of aligned edges includes: Identify lens status; Select the difference edge type combination based on the lens status; An edge belonging to the combination of differential edge types is instantiated from the set of aligned edges to form the first differential edge that needs to be reviewed.

4. The method according to claim 1, characterized in that, The first difference edge is subjected to multi-dimensional review to determine candidate difference edges, including: The first differential edge is subject to multi-dimensional review; The first difference edge that fails the review in any dimension is identified as a candidate difference edge.

5. The method according to claim 1, characterized in that, Identifying anomalous difference edges from the candidate difference edges includes: Calculate the anomaly detection index for the candidate difference edge; Candidate difference edges that satisfy the preset gating conditions are determined to be abnormal difference edges.

6. The method according to claim 1, characterized in that, After identifying the abnormal difference edges from the candidate difference edges, the process further includes: A rework contract is generated based on the abnormal differential edge and the evidence node associated with the abnormal differential edge; The video to be reviewed shall be repaired in accordance with the repair contract.

7. A video review device, characterized in that, The device includes: The first acquisition unit is used to acquire the video to be reviewed and its associated specified data; The second acquisition unit is used to obtain the set of normative constraints, the set of actual observations, and the set of evidence nodes based on the video to be reviewed and the specified data. A construction unit is used to write the set of normative constraints, the set of actual observations, and the set of evidence nodes into the audit graph to construct an alignment edge set; The first determining unit is used to determine the first differential edge from the set of aligned edges; The review unit is used to perform multi-dimensional review on the first difference edge to determine the candidate difference edge; The second determining unit is used to determine abnormal differential edges from the candidate differential edges, wherein the abnormal differential edges represent anomalies present in the video to be reviewed.

8. The apparatus according to claim 7, characterized in that, The second acquisition unit is specifically used for: Compile the specified data into a structured set of normative constraints; The video to be reviewed is analyzed to obtain the overall observation state, the character-scene occupancy field, and the contact-occlusion state sequence; Based on the observed states, the role-scene occupancy field, and the contact-occlusion state sequence, a set of actual observations and a set of evidence nodes are formed.

9. An electronic device, characterized in that, include: A processor and a memory are connected via a bus; wherein the processor is used to call and execute a program stored in the memory; The memory is used to store a program for implementing the video review method as described in any one of claims 1-6.

10. A storage medium, characterized in that, The storage medium stores computer-executable instructions for performing the video review method as described in any one of claims 1-6.

Images