A video authenticity identification method and system based on image classification

Abstract

The application relates to the technical field of image recognition and video content detection, and discloses a video authenticity identification method and system based on image classification, which comprises the following steps: acquiring a video to be detected and extracting candidate image frames, determining effective image frames according to face existence, face size, image definition, shielding proportion, compression strength and face posture angle; detecting and tracking face targets in the effective image frames, and associating face targets belonging to the same person as the same person track. The video authenticity identification method and system based on image classification can reduce candidate authenticity anomaly type misactivation caused by low-quality areas, shielding areas or compression distortion areas, reduce area evidence mismatch between face replacement anomalies, mouth shape driving anomalies, expression migration anomalies, whole face generation anomalies and local splicing anomalies, and improve the stability and reviewability of video-level authenticity anomaly detection conclusions.

Description

Technical Field

[0001] This invention relates to the field of image recognition and video content detection technology, and in particular to a method and system for identifying the authenticity of videos based on image classification. Background Technology

[0002] With the development of deep learning-based generation technology, videos exhibiting realism anomalies, such as face swapping, lip-syncing, expression transfer, full-face generation, and partial stitching, are becoming increasingly common. Existing video anomaly detection technologies typically begin by extracting frames from the video to be detected. Then, facial targets within the video frames are detected, cropped, and their features extracted. Subsequently, full-face images, partial facial feature images, facial boundary images, or multi-region images are input into an image classification model. The model outputs the probability of the true class or the probability of the real / abnormal class, and this is combined with frame-level scoring, region scoring, or time-series aggregation results to obtain a video-level authenticity detection conclusion.

[0003] However, existing technologies for determining video realism using multi-region image classification results typically focus on weighted fusion of region classification scores, inter-frame score smoothing, or temporal feature aggregation, lacking a mechanism to establish regional evidence relationships around specific realism anomaly types. Different realism anomaly types correspond to different anomaly regions. For example, lip-driven anomalies mainly manifest in the mouth and jaw regions, face-swapping anomalies are more likely to appear in the entire face area, face boundary areas, and extended background areas, while local splicing anomalies may be concentrated in local facial feature boundary areas. If detection conclusions are formed solely based on high classification probabilities of a single region, weighted multi-region score aggregation, or ordinary temporal aggregation, it is easy to mistakenly use anomalies from low-quality regions, occluded regions, or compressed distortion regions as the basis for establishing candidate realism anomaly types. It is also prone to evidence mismatches between types such as face-swapping anomalies, lip-driven anomalies, and expression transfer anomalies. Especially within the same person's trajectory, if the state constraints of candidate real anomaly types are not applied through support regions and counter-evidence regions, and delayed confirmation is performed in consecutive valid image frames, problems such as false activation of candidate real anomaly types, failure to eliminate regional evidence conflicts, and unstable video-level detection conclusions are likely to occur. Summary of the Invention

[0004] The technical problem to be solved by this invention is the shortcomings of existing technologies in video authenticity detection, such as region mismatch and type misjudgment. To address this, we propose a video authenticity identification method and system based on image classification.

[0005] To achieve the above objectives, this application adopts the following technical solution: a video authenticity identification method based on image classification, comprising: acquiring a video to be detected and extracting candidate image frames; determining effective image frames based on face presence, face size, image clarity, occlusion ratio, compression strength, and face pose angle; detecting and tracking face targets in the effective image frames; associating face targets belonging to the same person with the same person's trajectory; and generating multiple types of region images based on face bounding boxes and face key points within the same person's trajectory, wherein the multiple types of region images include images corresponding to the whole face region, local facial feature region, face boundary region, and face-extending background region; inputting the multiple types of region images into an image classification model to obtain the true category probability, multiple authenticity anomaly category probabilities, and region quality status of each region image, wherein the region quality status includes qualified status, low quality status, and unusable status; activating candidate authenticity anomaly types based on the authenticity anomaly category probabilities and region types of each region image in the same effective image frame; and calling the true category corresponding to the candidate authenticity anomaly type. The evidence set for real anomaly types includes a supporting region and a rebuttal region. The supporting region is used to confirm candidate real anomaly types, and the rebuttal region is used to identify region states that conflict with candidate real anomaly types. Based on whether the supporting region forms a region-coordinated anomaly, whether the rebuttal region meets the conflict condition, and the region quality state, the candidate real anomaly type is updated to a supporting state, a suppressed state, or a pending verification state. The region-coordinated anomaly is defined as at least two supporting regions meeting the anomaly condition, or one main supporting region meeting the high-confidence anomaly condition and at least one cooperating supporting region meeting the auxiliary anomaly condition. Within the same person's trajectory, the states of candidate real anomaly types are statistically analyzed chronologically. When the same candidate real anomaly type is continuously in a supporting state and not continuously set to a suppressed state in at least a preset number of consecutive valid image frames, and its supporting region continuously meets the anomaly condition, a suspected real anomaly segment is generated. A video-level detection conclusion for the video to be detected is generated based on the suspected real anomaly segment.

[0006] Preferably, the multiple types of region images include whole face region images, partial facial feature region images, face boundary region images, and face outward background region images; wherein, the partial facial feature region images correspond to one or more of the eyes, eyebrows, nose, mouth, or jaw, the face boundary region images are used to characterize the transition state between the face and the surrounding area, and the face outward background region images are used to characterize the consistency state between the face region and the surrounding background.

[0007] Preferably, the activation of candidate realism anomaly types includes: when the same realism anomaly category reaches a preset candidate threshold in a qualified region image, and the region type of the region image matches the supporting region corresponding to the realism anomaly category, the realism anomaly category is determined as a candidate realism anomaly type; when the candidate activation criterion comes only from a single region image in a low-quality state or an unavailable state, the candidate realism anomaly type is not activated.

[0008] Preferably, the support regions are configured according to the types of realism anomalies; wherein, the support regions for face-swapping anomalies include the whole face region, the face boundary region, and the face-extending background region; the support regions for lip-syncing anomalies include the mouth region and the jaw region; the support regions for expression migration anomalies include the eye region, the eyebrow region, and the mouth region; the support regions for whole face generation anomalies include the whole face region and the face-extending background region; and the support regions for local splicing anomalies include the face boundary region and the local facial feature boundary region.

[0009] Preferably, updating the candidate authenticity anomaly type to a support state, a suppression state, or a pending verification state includes: setting the candidate authenticity anomaly type to a support state when the support region forms a regional coordination anomaly and the counter-evidence region does not meet the conflict condition; setting the candidate authenticity anomaly type to a suppression state when the counter-evidence region points to another authenticity anomaly type and the support region does not form a regional coordination anomaly; and setting the candidate authenticity anomaly type to a pending verification state when the regional quality state of the support region or the counter-evidence region is a low-quality state or an unavailable state, or when the classification results of the support region and the counter-evidence region do not meet the preset consistency requirements within a preset number of consecutive valid image frames.

[0010] Preferably, generating suspected real anomaly segments includes: within the same person's trajectory, if the same candidate real anomaly type remains in a supported state in no less than a preset number of consecutive valid image frames and is not continuously set to a suppressed state, then a suspected real anomaly segment is generated; if two or more candidate real anomaly types remain in a supported state and the corresponding supported regions are different, then the corresponding time period is marked as a composite anomaly state, and a suspected real anomaly segment or a segment to be reviewed is generated based on the persistence of each candidate real anomaly type.

[0011] Preferably, before activating candidate real anomaly types, a pre-scan is performed on the pre-configured evidence groups for each real anomaly type; the pre-scan includes: reading the probability of real anomaly categories and the region quality status of each region image in the same valid image frame according to the supporting region and the counter-evidence region corresponding to each real anomaly type, and obtaining the evidence consistency status corresponding to each real anomaly type.

[0012] Preferably, obtaining the evidence consistency status corresponding to each real anomaly type includes: when the supporting region of a real anomaly type forms a regional co-existing anomaly and the counter-evidence region does not meet the conflict condition, the real anomaly type is marked as an active type; when the anomaly evidence of a real anomaly type comes only from a single region image in a low-quality state or an unavailable state, or when its counter-evidence region meets the conflict condition, the real anomaly type is not marked as an active type.

[0013] Preferably, the activation of candidate authenticity anomaly types includes: using the activatable type as a candidate authenticity anomaly type, and performing state delay confirmation on the candidate authenticity anomaly type within the same person trajectory; when the candidate authenticity anomaly type is continuously in a supported state in no less than a preset number of consecutive valid image frames, and is not continuously set to a suppressed state, a suspected authenticity anomaly segment is generated.

[0014] A video synthesis tampering detection system based on image classification, characterized in that the system comprises: a video processing module, used to acquire the video to be detected, extract candidate image frames, and determine valid image frames based on face presence, face size, image clarity, occlusion ratio, compression strength, and face pose angle; a trajectory and region generation module, used to detect and track face targets in the valid image frames, associate face targets belonging to the same person as the same person's trajectory, and generate multiple types of region images based on face bounding boxes and face key points within the same person's trajectory; an image classification module, used to input the multiple types of region images into an image classification model to obtain the true category probability, multiple true anomaly category probabilities, and region quality status of each region image; and a true anomaly type evidence status determination module, used to activate candidate images based on the true anomaly category probability and region type of each region image in the same valid image frame. The system identifies genuine anomaly types by calling the evidence group corresponding to the candidate genuine anomaly type, or by pre-scanning each pre-configured evidence group before activating the candidate genuine anomaly type. Based on whether the supporting region forms a regional collaborative anomaly, whether the counter-evidence region meets the conflict conditions, and the regional quality status, the candidate genuine anomaly type is updated to a supporting state, a suppressed state, or a pending verification state. The trajectory confirmation module is used to statistically analyze the status of candidate genuine anomaly types within the trajectory of the same person in chronological order. When the same candidate genuine anomaly type is continuously in a supporting state, not continuously set to a suppressed state, and the corresponding supporting region continuously meets the anomaly conditions in at least a preset number of consecutive valid image frames, a suspected genuine anomaly segment is generated. The video-level judgment module is used to generate a video-level detection conclusion for the video to be detected based on the suspected genuine anomaly segment.

[0015] The technical effects and advantages of this invention are as follows: This invention classifies images of multiple regions within the same person's trajectory and calls corresponding evidence groups based on candidate anomaly types. This eliminates the reliance on high classification probabilities of a single region or ordinary multi-region score fusion for determining candidate anomaly types. Instead, it is constrained by supporting regions, counter-evidence regions, and region quality states. Supporting regions confirm the validity of candidate anomaly types, counter-evidence regions identify region states conflicting with candidate anomaly types, and trajectory-based state delay confirmation determines whether the candidate anomaly type is consistently valid across consecutive valid image frames. Therefore, this invention reduces false activation of candidate anomaly types caused by low-quality regions, occluded regions, or compressed distortion regions, minimizes regional evidence mismatches between face-swapping anomalies, lip-syncing anomalies, expression migration anomalies, full-face generation anomalies, and local stitching anomalies, and improves the stability and verifiability of video-level anomaly detection conclusions. Attached Figure Description

[0016] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts: Figure 1 This is a schematic diagram of the overall structure of the video authenticity anomaly detection system of the present invention; Figure 2 This is a schematic diagram of the video authenticity anomaly detection method of the present invention; Figure 3 This is a schematic diagram illustrating the effective image frame selection and trajectory establishment of the same person in this invention; Figure 4 This is a schematic diagram illustrating the generation of multi-region images according to the present invention; Figure 5 This is a schematic diagram illustrating the image classification and candidate realism anomaly type activation of the present invention; Figure 6 This is a schematic diagram of the evidence group and status update for the authenticity anomaly type of the present invention; Figure 7 This is a schematic diagram illustrating the state delay confirmation within the trajectory of the same person according to the present invention; Figure 8 This is a schematic diagram illustrating the activation of candidate types in the pre-scanning of the evidence group of this invention. Detailed Implementation

[0017] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0018] This invention provides a method and system for identifying the authenticity of videos based on image classification, applicable to scenarios such as short video upload review, material library entry review, online video detection, digital human video detection, and AI-generated video detection. The authenticity anomalies include face-swapping anomalies, lip-syncing anomalies, expression transfer anomalies, whole-face generation anomalies, partial splicing anomalies, and other video content generated, replaced, driven, or spliced ​​using AI models to manipulate facial features.

[0019] Reference Figure 1-8 As shown, in one embodiment, the video authenticity identification method based on image classification includes the following processing steps: First, the video to be detected is parsed, and candidate image frames are extracted from the video; then, valid image frames that can participate in the detection are selected, and the trajectory of the same person is established; multiple types of region images are generated within the trajectory of the person, and image classification is performed on the region images; finally, based on the image classification results, candidate authenticity anomaly type activation, authenticity anomaly type evidence status determination, trajectory status delay confirmation, and video-level conclusion generation are performed.

[0020] Therefore, this embodiment does not directly conclude the authenticity of a video based on the classification results of a single frame image. Instead, it first organizes the same person target in the video into a continuous person trajectory, and then continuously confirms the status of regional evidence of authenticity anomalies within this continuous person trajectory. This approach reduces the impact of fluctuations in single-frame classification results, insufficient image quality in local areas, and the mixing of regional evidence of different authenticity anomalies on video-level detection conclusions.

[0021] After receiving the video to be tested, the system reads its file format, encoding type, frame rate, resolution, video duration, total number of frames, and keyframe information. If the video is corrupted, cannot be decoded, has missing frames, or has abnormal encoding information, the system marks it as a video to be reviewed and records the reason for the anomaly. When the video can be parsed normally, the system creates a real anomaly detection task and enters the candidate image frame extraction process. For videos shorter than a preset duration threshold, the system extracts candidate image frames at fixed time intervals. For videos longer than or equal to the preset duration threshold, the system first divides the video into segments based on the degree of image change between adjacent frames, and then extracts candidate image frames from each video segment. The degree of image change can be determined by color histogram changes, structural similarity changes, edge distribution changes, or image feature distance; in practice, one method or a combination of these can be used.

[0022] After candidate image frames are extracted, the system evaluates their validity. The evaluation includes face presence, face size, image sharpness, exposure, occlusion ratio, compression strength, and face pose angle. If no face is detected in a candidate image frame, or the face size is below a preset size threshold, or the image sharpness is below a preset sharpness threshold, or the occlusion ratio of the face region is above a preset occlusion threshold, the system does not classify the candidate image frame as valid. For candidate image frames with low-quality regions but located within a continuous human trajectory, the system marks them as auxiliary frames. Auxiliary frames are not used as the primary basis for generating candidate real anomaly types, but are used to maintain the continuity of the human trajectory or to help determine the temporal boundaries of suspected real anomaly segments.

[0023] The system also generates a region quality status for the face region and its corresponding local region image in each valid image frame. The region quality status is divided into three categories: acceptable, low-quality, and unusable. Region images in the acceptable state can be used for candidate anomaly type determination; region images in the low-quality state can be used as auxiliary references but are not used alone to trigger a suspected authenticity conclusion; region images in the unusable state do not participate in candidate anomaly type determination. In specific implementations, the region quality status can be determined based on the region image's sharpness, occlusion ratio, effective pixel ratio, exposure status, compression intensity, and pose shift degree.

[0024] After obtaining valid image frames, the system performs face detection on each valid image frame to obtain face bounding boxes and facial key points. These facial key points can cover the eyes, eyebrows, nose, mouth, facial contours, and jaw. In cases where multiple face targets exist within the same valid image frame, the system generates a target number for each face target and associates face targets belonging to the same person with the same person's trajectory based on the overlap, size variation, facial feature similarity, and key point geometric relationships of face targets in adjacent valid image frames.

[0025] Each person's trajectory record includes a trajectory identifier, start time, end time, the sequence number of the valid image frames it contains, and the location of the face target in each valid image frame. After generating the region image, the system then associates the quality status of the corresponding region with that person's trajectory. For multi-person face videos, the system performs a subsequent authenticity anomaly type evidence status determination for each person's trajectory to avoid mutual interference between the face region detection results of different people.

[0026] For each valid image frame within the same person's trajectory, the system generates multiple types of region images based on the face bounding box and facial key points. These region images are categorized as: whole face region image, eye region image, eyebrow region image, nose region image, mouth region image, jaw region image, face boundary region image, face-extending background region image, and local facial feature boundary region image. These region images are used to represent the overall face, local facial features, boundary transitions, and background consistency, respectively.

[0027] The image is divided into several parts: the full-face image, obtained by expanding the face bounding box according to a preset expansion ratio, and the full-face generation traces. The eye area image is obtained by cropping the key points of both eyes and their preset expansion range, and is used to reflect periorbital texture, eyelid edges, pupil reflection, and abnormal eye shape. The eyebrow area image is obtained by cropping the key points of the eyebrows and their preset expansion range, and is used to reflect eyebrow movement and expression migration traces. The nose area image is obtained by cropping the key points of the nose and their preset expansion range, and is used to reflect nasal texture, nasal alar edges, and the transition between the nose and the mid-face area. The mouth area image is obtained by cropping the key points of the lips and their preset expansion range, and is used to reflect lip edges, teeth area, oral cavity shadows, and abnormal mouth deformation. The jaw area image is obtained by cropping the key points of the jaw and the lower boundary of the face, and is used to reflect the jaw contour and facial movement deformation. The face boundary area image is obtained by the ring-shaped area formed by the facial contour key points extending inwards and outwards, and is used to reflect the fusion traces between the face and hair, ears, and neck. The image of the extended background region of the face is obtained by expanding the face bounding box outwards and excluding the main face area. It is used to reflect the consistency of lighting, noise, and compressed texture between the face area and the surrounding background. The image of the local facial feature boundary region is a transition area formed by expanding the edges of one or more regions in the eye region, eyebrow region, nose region, or mouth region inwards and outwards by a preset width. It is used to reflect the splicing marks, texture breaks, and boundary transition anomalies between the local facial features and adjacent facial regions.

[0028] Subsequently, the system performs size normalization, brightness normalization, color normalization, and region masking on each region image, and then inputs the processed region images into an image classification model. The image classification model can employ a convolutional neural network, a visual Transformer network, a classification network combining convolutional and attention networks, or an image classification model with the same output format. Different region images can be input into the same multi-class classification model or into separately trained region classification models. The image classification model outputs the true class probability and multiple realism anomaly category probabilities for each region image. The realism anomaly category probabilities correspond to face-swapping anomalies, lip-syncing anomalies, expression transfer anomalies, full-face generation anomalies, and local stitching anomalies. Based on the classification results of each region image, the system generates a region classification record, which records the person trajectory identifier, frame number, region type, region quality status, true class probability, and probabilities of each realism anomaly category.

[0029] Based on this, the system determines candidate real anomaly types according to the classification results of multiple regions in the same valid image frame. The system pre-configures the supporting regions and counter-evidence regions corresponding to each real anomaly type, and reads the probability of the real anomaly category corresponding to each region image to determine whether there are real anomaly categories that meet the candidate activation conditions. The candidate activation conditions can be set as follows: the probability of the same real anomaly category in a qualified region image reaches a preset candidate threshold; the same real anomaly category reaches the preset candidate threshold in multiple region images; or the same real anomaly category reaches the preset candidate threshold within a pre-configured supporting region. When a real anomaly category meets the candidate activation conditions, the system determines that real anomaly category as a candidate real anomaly type in the current valid image frame; if multiple real anomaly categories simultaneously meet the candidate activation conditions, the system retains multiple candidate real anomaly types and performs subsequent evidence status determinations respectively; if no candidate real anomaly type is formed, the system records the valid image frame as inactive and continues processing the next valid image frame. Therefore, the formation of candidate real anomaly types does not simply depend on the high classification probability of a certain region, but also requires consideration of whether the region type corresponding to that high classification probability matches the pre-configured supporting regions. During the activation process of candidate real anomaly types, if the candidate activation condition is triggered by only a single low-quality region, the system will not directly determine the corresponding real anomaly category as the candidate real anomaly type, but will use the classification result of that region as the basis for review.

[0030] The system can perform a pre-scan of evidence groups before activating candidate anomaly types. During the pre-scan, the system pre-scans the region classification results in the same valid image frame according to the supporting regions and counter-evidence regions corresponding to each anomaly type, and generates the evidence consistency state for each anomaly type. When the supporting regions of a certain anomaly type form a region co-anomaly, the quality state of the supporting regions involved in the judgment meets the preset requirements, and the counter-evidence regions of the anomaly type do not meet the conflict conditions, the system marks the anomaly type as an activatable type. Conversely, if a certain anomaly type has a high classification probability only in a single low-quality region, or its counter-evidence regions have already met the conflict conditions, the system does not mark the anomaly type as an activatable type. Subsequently, the candidate anomaly type activation unit determines the candidate anomaly type based on the activatable types obtained from the evidence group pre-scan. Through this processing method of scanning the evidence structure first and then activating the candidate types, the system can reduce false activations caused by low-quality regions, occluded regions, compression distortion regions, or regions that do not match the anomaly type.

[0031] After identifying a candidate true anomaly type, the system calls the corresponding evidence group for that candidate true anomaly type and updates its state based on this evidence group. The true anomaly type evidence group consists of a supporting region and a rebuttal region. The supporting region is used to determine whether the candidate true anomaly type has valid grounds, while the rebuttal region is used to determine whether there is a type conflict. It should be noted that the anomaly result of the rebuttal region does not necessarily negate the candidate true anomaly type; its role is to identify region states that do not match the candidate true anomaly type. When the classification result of the rebuttal region points to another true anomaly type, and the supporting region of the candidate true anomaly type does not form a region-coordinated anomaly or maintain a supporting state in consecutive valid image frames, the system sets the candidate true anomaly type to a suppressed state. Alternatively, when the anomaly state of the rebuttal region conflicts with the evidence structure corresponding to the candidate true anomaly type, the system also sets the candidate true anomaly type to a suppressed state.

[0032] Different types of realism anomalies correspond to different evidence set structures. Face-swapping anomalies use the entire face region, face boundary region, and extended background region as support regions, and the mouth region and jaw region as counter-evidence regions. Lip-shape driven anomalies use the mouth region and jaw region as support regions, and the face boundary region and extended background region as counter-evidence regions. Expression transfer anomalies use the eye region, eyebrow region, and mouth region as support regions, and the face boundary region as counter-evidence region. Full-face generation anomalies use the entire face region, eye region, and extended background region as support regions, and the local facial feature boundary region as counter-evidence region. Local splicing anomalies use the face boundary region and local facial feature boundary regions as support regions, and the entire face region as counter-evidence region. These evidence sets can be adjusted based on training samples or preset detection scenarios, but the adjusted evidence sets still maintain the correspondence between support regions for confirming candidate realism anomaly types and counter-evidence regions for identifying type conflicts.

[0033] The system reads the classification results and region quality status of the supporting and counter-evidence regions corresponding to the candidate real anomaly types. Based on whether the supporting regions form regional co-anomalies, whether the counter-evidence regions meet the conflict conditions, and the region quality status, the system updates the status of the candidate real anomaly types. The candidate real anomaly type status is divided into support status, suppression status, and pending verification status. Regional co-anomalies can be formed in two ways: one is that at least two supporting regions meet the anomaly conditions; the other is that a primary supporting region meets the high-confidence anomaly condition, while at least one cooperating supporting region meets the auxiliary anomaly condition. The primary and cooperating supporting regions are pre-defined according to the real anomaly type, and the classification probability threshold corresponding to the high-confidence anomaly condition is higher than the classification probability threshold corresponding to the auxiliary anomaly condition. The anomaly condition can be set to the classification probability of the candidate real anomaly type corresponding to the supporting region reaching a first anomaly threshold, or it can be set to the classification probability of the same candidate real anomaly type for the supporting region reaching a preset trend threshold in consecutive valid image frames.

[0034] If a supporting region exhibits a regional coordination anomaly, and the supporting region's regional quality status is satisfactory, and the counter-evidence region does not meet the conflict condition, the system sets the candidate real anomaly type to a supporting state. If the counter-evidence region meets the conflict condition, the system sets the candidate real anomaly type to a suppressed state. If the supporting region or the counter-evidence region's regional quality status is low-quality or unavailable, or if the classification results of the supporting region and the counter-evidence region switch more than a preset consistency threshold within a preset number of consecutive valid image frames, or if the supporting region does not exhibit a regional coordination anomaly, the system sets the candidate real anomaly type to a pending review state. If the supporting region and the counter-evidence region point to different real anomaly types, and both meet the corresponding anomaly conditions in consecutive valid image frames, the system marks the corresponding segment as a composite anomaly segment or a segment pending review.

[0035] The system also performs type mutual exclusion determination based on the support and suppression states of candidate realism anomaly types. Type mutual exclusion determination is used to reduce confusion among realism anomaly types when multiple candidate realism anomaly types are activated simultaneously. For example, when the mouth region and jaw region meet the anomaly conditions in consecutive valid image frames, and the face boundary region and the extended background region do not meet the conflict conditions, the system prioritizes supporting lip-sync anomalies and suppresses face-swapping anomalies. When the entire face region, face boundary region, and extended background region simultaneously meet the anomaly conditions, and the mouth region does not meet the anomaly conditions corresponding to lip-sync anomalies, the system prioritizes supporting face-swapping anomalies or whole-face generation anomalies based on the dominant anomaly region among the entire face region, face boundary region, and extended background region. Specifically, when the fusion anomaly between the face boundary region and the extended background region reaches a preset fusion anomaly threshold, face-swapping anomalies are prioritized; when the overall texture anomaly between the entire face region and the extended background region reaches a preset texture anomaly threshold, whole-face generation anomalies are prioritized. When the eye, eyebrow, and mouth regions simultaneously meet the abnormal conditions, but the fusion anomaly in the face boundary region does not reach the preset fusion anomaly threshold, the system prioritizes supporting expression transfer anomalies. When anomalies are concentrated in local facial feature boundary regions, and the entire face region does not meet the abnormal conditions corresponding to whole-face generation anomalies, the system prioritizes supporting local stitching anomalies. When anomalies originate only from a single region with a low quality state, a region with an occlusion ratio higher than the preset occlusion threshold, or a region with a compression intensity higher than the preset compression threshold, the system does not directly generate suspected real-world anomaly segments and marks the candidate real-world anomaly type corresponding to the valid image frame as pending review.

[0036] When two or more candidate realism anomaly types are supported in consecutive valid image frames, and their corresponding supporting regions are different, and the counter-evidence regions do not meet the conflict conditions, the system marks them as a composite anomaly state. Composite anomaly states do not directly cancel the suspected realism judgment; instead, they are counted separately in the subsequent state delay confirmation within the trajectory. This is to accommodate situations where multiple realism methods, such as face swapping, lip-syncing, and / or local stitching, exist simultaneously in the same video. By setting composite anomaly states, the system can retain processing paths for multiple types of anomalies coexisting, reducing erroneous judgments caused by forcibly selecting a single realism anomaly type.

[0037] The system reads the candidate authenticity anomaly type status of each valid image frame within the same person's trajectory in chronological order and performs a status delay confirmation on the candidate authenticity anomaly type. The system generates a suspected authenticity anomaly segment only when at least a preset number of consecutive valid image frames within the same person's trajectory meet the following conditions: the same candidate authenticity anomaly type is continuously in a supported state; the candidate authenticity anomaly type is not continuously set to a suppressed state; the supporting region corresponding to the candidate authenticity anomaly type meets the anomaly conditions in consecutive valid image frames; and the counter-evidence region does not continuously generate a state that conflicts with the candidate authenticity anomaly type. The suspected authenticity anomaly segment records the person's trajectory identifier, start time, end time, candidate authenticity anomaly type, supporting region, counter-evidence region, and anomaly region location. When the same candidate authenticity anomaly type is only in a supported state in a single valid image frame, or the number of times the candidate authenticity anomaly type switches within a preset time window exceeds a preset switching threshold, or the counter-evidence region continuously triggers conflict conditions in consecutive valid image frames, or the regional quality status of the supporting region does not reach a qualified state in consecutive valid image frames, the system does not generate a suspected authenticity anomaly segment and marks the corresponding time period as a segment to be reviewed.

[0038] State delay confirmation is achieved through continuous frame counting. The system sets a state counter for each candidate realism anomaly type in each character's trajectory. When a candidate realism anomaly type is in a supported state and not in a suppressed state, the support counter increases; when a candidate realism anomaly type is in a suppressed state or a pending verification state, the support counter is reset to zero or decreased according to a preset decay rule; when the support counter reaches a preset number of consecutive frames, the system generates a suspected realism anomaly segment. This state counter is only used to record state changes and is not limited to a specific mathematical calculation method. For composite anomaly states, the system separately counts the persistence of each candidate realism anomaly type within the same character's trajectory. If at least one candidate realism anomaly type meets the conditions for generating a suspected realism anomaly segment, the system generates a suspected realism anomaly segment and marks it as a composite anomaly in the segment attributes; if none of the candidate realism anomaly types meet the persistence conditions, the system marks the corresponding time period as a pending verification segment.

[0039] The system generates video-level detection conclusions based on suspected anomaly segments in each person's trajectory. If no suspected anomaly segments meet the conditions in the video to be detected, and the number of valid image frames is not less than a preset valid frame count threshold, the system outputs a true video conclusion. If at least one suspected anomaly segment exists in the video to be detected, and the duration of the suspected anomaly segment is not less than a preset segment duration threshold, and the stability of the candidate anomaly type and the stability of the supporting region both meet preset requirements, the system outputs a suspected anomaly video conclusion. If the number of valid image frames in the video to be detected is less than a preset valid frame count threshold, the video quality is lower than a preset video quality threshold, the target occlusion ratio is higher than a preset occlusion threshold, the number of times the candidate anomaly type switches within a preset time window exceeds a preset switching threshold, there is a conflict in the detection conclusions between different person trajectories, or the number or duration of segments marked as pending review exceeds a preset review ratio threshold, the system outputs a video conclusion pending review.

[0040] For multi-person face videos, the system first generates detection conclusions for the trajectories of each person separately, and then generates a detection conclusion for the entire video based on the suspected abnormal segments corresponding to each person's trajectory. If any person's trajectory is determined to be suspected of being abnormal, the system classifies the video to be tested as a suspected abnormal video. If multiple person trajectory conclusions with a duration not less than a preset trajectory duration threshold or a screen proportion not less than a preset proportion threshold conflict with each other, the system outputs a video conclusion to be reviewed and lists the independent detection results of each person's trajectory in the detection report.

[0041] A video synthesis tampering detection system based on image classification is provided for executing the above method. The system includes a video access module, a video parsing module, a candidate image frame extraction module, a valid image frame filtering module, a person trajectory establishment module, a region image generation module, an image classification module, a authenticity anomaly type evidence status determination module, a video-level determination module, and a detection report generation module.

[0042] The video access module receives the video to be inspected and generates an inspection task record. The video parsing module reads the file format, encoding type, frame rate, resolution, video duration, total number of frames, and keyframe information of the video to be inspected, and generates a check mark when the video to be inspected cannot be parsed normally. The candidate image frame extraction module extracts candidate image frames from the video to be inspected based on time intervals, keyframe positions, or the degree of image change. The effective image frame filtering module filters effective image frames based on face presence, face size, image clarity, exposure status, occlusion ratio, compression strength, and face pose angle, and generates the face region quality status of the effective image frames.

[0043] The character trajectory establishment module performs face detection and facial landmark detection on valid image frames. Based on the overlap, size variation, facial feature similarity, and geometric relationships of key points among adjacent valid image frames, it associates facial targets of the same person with the same character trajectory. The region image generation module generates full-face region images, eye region images, eyebrow region images, nose region images, mouth region images, jaw region images, face boundary region images, face background region images, and local facial feature boundary region images based on face bounding boxes and facial landmarks. After generating each region image, it associates the corresponding region quality status with each region image. The image classification module preprocesses and classifies each region image, outputting the probability of the true class, the probability of face-swapping anomalies, the probability of lip-sync anomalies, the probability of expression transfer anomalies, the probability of full-face generation anomalies, and the probability of local stitching anomalies.

[0044] The authenticity anomaly type evidence state determination module includes a candidate authenticity anomaly type activation unit, an evidence group invocation unit, a state update unit, a type mutual exclusion determination unit, and a trajectory-based state delay confirmation unit. The candidate authenticity anomaly type activation unit determines candidate authenticity anomaly types based on the probability of authenticity anomaly categories in multiple regional images, and when the candidate activation condition is triggered only by a single low-quality region, the corresponding region classification result is used as the basis for verification. The evidence group invocation unit invokes the corresponding authenticity anomaly type evidence group based on the candidate authenticity anomaly type. The authenticity anomaly type evidence group consists of supporting regions and counter-evidence regions. The state update unit updates the candidate authenticity anomaly type to a supporting state, a suppressed state, or a state awaiting verification based on whether the supporting region forms a regional collaborative anomaly, whether the counter-evidence region meets the conflict condition, and the region quality state. When multiple candidate authenticity anomaly types are activated simultaneously, the type mutual exclusion determination unit corrects the candidate authenticity anomaly types based on the states of the supporting and counter-evidence regions; when two or more candidate authenticity anomaly types are in a supporting state in consecutive valid image frames, and the counter-evidence region does not meet the conflict condition, the type mutual exclusion determination unit generates a composite anomaly state. The trajectory-in-trajectory state delay confirmation unit continuously counts the status of candidate real anomaly types within the same person's trajectory. When the same candidate real anomaly type is continuously in a support state, not continuously in a suppression state, the support region remains stable and the counter-evidence region does not continuously conflict, a suspected real anomaly segment is generated. For composite anomaly states, the trajectory-in-trajectory state delay confirmation unit counts the persistence of each candidate real anomaly type and generates a suspected real anomaly segment or a segment to be reviewed based on the statistical results.

[0045] The evidence status determination module for genuine anomaly types also includes an evidence group pre-scanning unit. Before the activation of candidate genuine anomaly types, the evidence group pre-scanning unit pre-scans the region classification results in the same valid image frame according to the supporting region and the rebuttal region corresponding to each genuine anomaly type, and generates an evidence consistency status for each genuine anomaly type. The candidate genuine anomaly type activation unit determines the activatable type based on the evidence consistency status and uses the activatable type as the candidate genuine anomaly type. This setting can reduce the false activation of candidate types caused by a high classification probability of a single region.

[0046] The video-level judgment module generates video-level detection conclusions based on suspected abnormal segments in the trajectories of each person. If no suspected abnormal segments meet the conditions and the number of valid image frames is not less than a preset valid frame count threshold, a true video conclusion is output. If suspected abnormal segments meet the conditions, a suspected abnormal video conclusion is output. If the number of valid image frames is less than a preset valid frame count threshold, the video quality is lower than a preset video quality threshold, the target occlusion ratio is higher than a preset occlusion threshold, the number of times the candidate abnormal type switches within a preset time window exceeds a preset switching threshold, there is a conflict in the detection conclusions between different person trajectories, or the number or duration of segments marked as pending review exceeds a preset review ratio threshold, a pending review video conclusion is output. The detection report generation module generates a detection report based on the video-level detection conclusions. The detection report records basic video information, detection conclusions, target person identifiers, suspected abnormal time periods, candidate abnormal types, supporting regions, counter-evidence regions, candidate abnormal type status, composite abnormal status, abnormal region annotation map, abnormal frame screenshots, and review suggestions.

[0047] The video authenticity anomaly detection task can be executed asynchronously through a task queue. The system writes the task identifier, video address, detection parameters, and callback address into the task queue. After reading the task, the detection node sequentially executes the following steps: video parsing, candidate image frame extraction, effective image frame filtering, person trajectory establishment, region image generation, image classification, candidate authenticity anomaly type activation, authenticity anomaly type evidence status determination, trajectory state delay confirmation, video-level determination, and detection report generation. The detection parameters can be selected from the candidate frame extraction interval, effective frame filtering threshold, candidate activation threshold, and number of consecutive frames. If video download failure, decoding failure, image classification model call failure, or detection report generation failure occurs during the detection process, the system records the anomaly information and re-executes the detection task according to the preset retry rules. If the detection still cannot be completed after retrying, the system updates the task status to detection failure and saves the reason for the failure. After the detection is completed, the system writes the video-level detection conclusion, suspected authenticity anomaly segments, and detection report address back to the database.

[0048] The image classification model is trained using real video samples and real-world anomalous video samples, with no specific training framework limited. During training, the system extracts image frames from the sample videos and generates whole-face regions, eye regions, eyebrow regions, nose regions, mouth regions, jaw regions, face boundary regions, extended background regions, and local facial feature boundary regions based on facial key points. Each region image is labeled according to sample annotations as a real category, a face-swapping anomalous category, a lip-syncing anomalous category, an expression transfer anomalous category, a whole-face generation anomalous category, or a local splicing anomalous category. During training, the region images can be compressed, scaled, blurred, have brightness variations, simulate occlusion, and be cropped for enhancement, allowing the image classification model to adapt to videos of different resolutions, compression levels, and platform sources.

[0049] The system writes the results of manual review into the sample database. For videos that are manually identified as false positives or false negatives, the system saves the corresponding video clips, abnormal image frames, region images, review conclusions, and the category of authenticity anomalies, and uses these as samples for subsequent model retraining or incremental training. After the updated model is validated, it is deployed to the detection service nodes to perform subsequent detection tasks.

[0050] Through the above implementation methods, this invention converts the multi-region image classification results in video authenticity anomaly detection into candidate authenticity anomaly type states within the trajectory of the same person. By activating candidate authenticity anomaly types, calling authenticity anomaly type evidence groups, updating the states of supporting and counter-evidence regions, determining type mutual exclusion, handling composite anomalies, and confirming delayed states within the trajectory, this invention avoids relying solely on high classification probabilities of a single frame or ordinary multi-region fusion to directly form video-level conclusions, thereby improving the stability of authenticity anomaly video detection and the interpretability of verification.

[0051] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A method for identifying the authenticity of videos based on image classification, characterized in that, include: The process involves acquiring a video to be detected and extracting candidate image frames. Valid image frames are determined based on face presence, face size, image clarity, occlusion ratio, compression strength, and face pose angle. Face targets within the valid image frames are detected and tracked. Face targets belonging to the same person are associated with the same person's trajectory. Multiple region images are generated based on the face bounding boxes and facial key points within the same person's trajectory. These multiple region images include images corresponding to the whole face region, local facial feature region, face boundary region, and the background region extending beyond the face. The multiple region images are input into an image classification model to obtain the true category probability, multiple true anomaly category probabilities, and region quality status for each region image. The region quality status includes acceptable, low-quality, and unusable states. Based on the true anomaly category probabilities and region types of each region image in the same valid image frame, candidate true anomaly types are activated. The true anomaly type evidence group corresponding to the candidate true anomaly type is invoked. The true anomaly type evidence group includes... The system includes a support region and a counter-evidence region. The support region is used to confirm the candidate real anomaly type, and the counter-evidence region is used to identify the region state that conflicts with the candidate real anomaly type. Based on whether the support region forms a region-coordinated anomaly, whether the counter-evidence region meets the conflict condition, and the region quality state, the candidate real anomaly type is updated to a support state, a suppression state, or a pending verification state. The region-coordinated anomaly is defined as at least two support regions meeting the anomaly condition, or one main support region meeting the high-confidence anomaly condition and at least one cooperating support region meeting the auxiliary anomaly condition. Within the same person's trajectory, the states of candidate real anomaly types are counted sequentially over time. When the same candidate real anomaly type is continuously in a support state and not continuously set to a suppression state in at least a preset number of consecutive valid image frames, and its support region continuously meets the anomaly condition, a suspected real anomaly segment is generated. A video-level detection conclusion for the video to be detected is generated based on the suspected real anomaly segment.

2. The video authenticity verification method based on image classification according to claim 1, characterized in that, The multiple types of region images include whole face region images, partial facial feature region images, face boundary region images, and face outward background region images; wherein, the partial facial feature region images correspond to one or more of the eyes, eyebrows, nose, mouth, or jaw, the face boundary region images are used to characterize the transition state between the face and the surrounding area, and the face outward background region images are used to characterize the consistency state between the face region and the surrounding background.

3. The video authenticity verification method based on image classification according to claim 2, characterized in that, The activation of candidate realism anomaly types includes: when the same realism anomaly category reaches a preset candidate threshold in a qualified region image, and the region type of the region image matches the supporting region corresponding to the realism anomaly category, the realism anomaly category is determined as a candidate realism anomaly type; when the candidate activation criterion comes only from a single region image in a low-quality state or an unavailable state, the candidate realism anomaly type is not activated.

4. The video authenticity verification method based on image classification according to claim 3, characterized in that, The support regions are configured according to the types of realism anomalies; among them, the support regions for face-swapping anomalies include the whole face region, the face boundary region, and the face-extending background region; the support regions for mouth-shaped driving anomalies include the mouth region and the jaw region; the support regions for expression transfer anomalies include the eye region, the eyebrow region, and the mouth region; the support regions for whole face generation anomalies include the whole face region and the face-extending background region; and the support regions for local splicing anomalies include the face boundary region and the local facial feature boundary region.

5. The video authenticity verification method based on image classification according to claim 4, characterized in that, The step of updating the candidate authenticity anomaly type to a support state, a suppression state, or a pending review state includes: setting the candidate authenticity anomaly type to a support state when the support region forms a regional coordination anomaly and the counter-evidence region does not meet the conflict condition; setting the candidate authenticity anomaly type to a suppression state when the counter-evidence region points to another authenticity anomaly type and the support region does not form a regional coordination anomaly; and setting the candidate authenticity anomaly type to a pending review state when the regional quality state of the support region or the counter-evidence region is a low-quality state or an unavailable state, or when the classification results of the support region and the counter-evidence region do not meet the preset consistency requirements within a preset number of consecutive valid image frames.

6. The video authenticity verification method based on image classification according to claim 5, characterized in that, The generation of suspected real-world anomaly segments includes: within the trajectory of the same person, if the same candidate real-world anomaly type remains supported in a supported state in no less than a preset number of consecutive valid image frames and is not continuously set to a suppressed state, then a suspected real-world anomaly segment is generated; if two or more candidate real-world anomaly types remain supported and the corresponding supported regions are different, then the corresponding time period is marked as a composite anomaly state, and a suspected real-world anomaly segment or a segment to be reviewed is generated based on the persistence of each candidate real-world anomaly type.

7. The video authenticity verification method based on image classification according to claim 1, characterized in that, Before activating candidate real anomaly types, a pre-scan is performed on the pre-configured evidence groups for each real anomaly type. The pre-scan includes: reading the probability of real anomaly categories and the quality status of regions in the same valid image frame according to the supporting region and the counter-evidence region corresponding to each real anomaly type, so as to obtain the evidence consistency status corresponding to each real anomaly type.

8. The video authenticity verification method based on image classification according to claim 7, characterized in that, The process of obtaining the evidence consistency status corresponding to each type of real anomaly includes: when the supporting region of a certain type of real anomaly forms a regional co-existence anomaly and the counter-evidence region does not meet the conflict condition, the real anomaly type is marked as an active type; when the anomaly evidence of a certain type of real anomaly comes only from a single region image in a low-quality state or an unavailable state, or when its counter-evidence region meets the conflict condition, the real anomaly type is not marked as an active type.

9. The video authenticity verification method based on image classification according to claim 8, characterized in that, The activation of candidate authenticity anomaly types includes: using the activatable type as a candidate authenticity anomaly type, and performing state delay confirmation on the candidate authenticity anomaly type within the same person trajectory; when the candidate authenticity anomaly type is continuously in a supported state in no less than a preset number of consecutive valid image frames, and is not continuously set to a suppressed state, a suspected authenticity anomaly segment is generated.

10. A video authenticity verification system based on image classification, characterized in that, The system is used to execute the video authenticity identification method based on image classification as described in any one of claims 1-9. The system includes: a video processing module for acquiring the video to be detected, extracting candidate image frames, and determining valid image frames based on face presence, face size, image clarity, occlusion ratio, compression strength, and face pose angle; a trajectory and region generation module for detecting and tracking face targets in the valid image frames, associating face targets belonging to the same person as the same person's trajectory, and generating multiple types of region images based on face bounding boxes and face key points within the same person's trajectory; an image classification module for inputting the multiple types of region images into an image classification model to obtain the true category probability, multiple authenticity anomaly category probabilities, and region quality status of each region image; and an authenticity anomaly type evidence status determination module for determining the authenticity anomaly category probability and region quality status of each region image within the same valid image frame. The system activates candidate real anomaly types by calling the corresponding evidence group for the candidate real anomaly type, or by pre-scanning each pre-configured evidence group for real anomaly types before activating the candidate real anomaly type. Based on whether the supporting region forms a regional collaborative anomaly, whether the counter-evidence region meets the conflict conditions, and the regional quality status, the candidate real anomaly type is updated to a supporting state, a suppressed state, or a pending verification state. The trajectory confirmation module is used to statistically analyze the status of candidate real anomaly types in chronological order within the trajectory of the same person. When the same candidate real anomaly type is continuously in a supporting state, not continuously set to a suppressed state, and the corresponding supporting region continuously meets the anomaly conditions in at least a preset number of consecutive valid image frames, a suspected real anomaly segment is generated. The video-level judgment module is used to generate a video-level detection conclusion for the video to be detected based on the suspected real anomaly segment.

Images