Video deduplication method and system based on key frame weighted feature fusion and medium

Abstract

The application discloses a video deduplication method and system based on key frame weighted feature fusion and a medium, and relates to the technical field of video content processing. The method comprises the following steps: acquiring a to-be-processed video and performing shot segmentation, and selecting representative key frames in each shot segment; generating a main body core according to the representative key frames and forming a disturbance envelope band outward, identifying a non-main body interference area and determining an interference intrusion state; screening main judgment key frames, extracting local texture features, semantic content features and effective picture global features, and performing weighted fusion to generate video-level candidate recall features; recalling candidate videos, establishing shot matching edges and shot closed chains, and outputting video deduplication results. By distinguishing between main body content and peripheral interference content, combining shot sequence continuous verification, and adopting candidate recall and closed chain verification for processing, the identification accuracy of repeated videos and near-repeated videos is improved, and the screening efficiency and judgment accuracy in a large-scale video library are taken into account.

Description

Technical Field

[0001] This invention relates to the field of video content processing technology, specifically to a video deduplication method, system, and medium based on keyframe weighted feature fusion. Background Technology

[0002] With the rapid development of short video platforms, advertising platforms, media asset management platforms, and content monitoring platforms, the number of video materials continues to grow. The same original video often appears repeatedly with different resolutions, bitrates, container formats, and platform packaging. Some videos are also overlaid with subtitles, watermarks, logos, borders, or overlays during dissemination, or republished by replacing intros and outros, partially cropping, recompressing, or splicing. For businesses such as material archiving, copyright monitoring, content review, ad deduplication, and media asset management, accurately identifying duplicate or near-duplicate videos has become a crucial technical challenge in the video processing field.

[0003] Current video deduplication techniques mainly employ whole-video hash comparison, fixed-interval frame-by-frame comparison, or similarity retrieval based on single-frame features. While whole-video hash comparison has lower computational cost, it is highly sensitive to video recoding, partial cropping, added visual interference, and rewriting of the beginning and end of the video, easily leading to the inaccurate recall of videos from the same source. Fixed-interval frame-by-frame comparison, while reducing the computational cost of whole-video frame-by-frame comparison to some extent, suffers from insufficient representativeness due to the lack of a direct correlation between the extracted frame positions and the actual shot structure, often resulting in the extraction of transition frames, transitional frames, or frames with a high proportion of interfering content, thus affecting the stability of the deduplication results. Retrieval methods based on single-frame or discrete keyframe features, while improving local content comparison capabilities, often struggle to effectively distinguish between the main content and peripheral interference in videos with subtitles, watermarks, station logos, or platform packaging, leading to misjudgments.

[0004] Furthermore, existing technologies typically focus on judging the overall similarity of videos or the similarity of local frames, while insufficiently considering the sequential relationships of shots and the continuous content structure within the video. In practical applications, montage videos, spliced ​​video clips, or partially borrowed videos may have high similarity to the original video in several shots, but they are not considered duplicate videos overall. Conversely, while some videos from the same source may have replaced or supplemented content at the beginning and end, the main content in the middle still maintains strong continuity. Existing technologies lack a comprehensive solution that can make a judgment based on the identification of the main content, combined with the degree of intrusion of interfering content and the closed-loop relationship of shot continuity. Therefore, it is difficult to simultaneously ensure the accuracy of identifying complete duplicate videos, videos with altered beginnings and ends from the same source, and partially overlapping videos. Summary of the Invention

[0005] Based on the shortcomings of the prior art described above, the purpose of this invention is to provide a video deduplication method, system, and medium based on keyframe weighted feature fusion to solve the aforementioned technical problems.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a video deduplication method based on keyframe weighted feature fusion, comprising: The video to be processed is acquired and segmented into shots. Representative keyframes that meet the preset conditions for clarity and stability are selected from each shot segment. The main kernel is generated based on the content regions that maintain structural stability after position compensation in consecutive adjacent frames in the representative keyframe. The perturbation envelope is formed by expanding outward in layers based on the boundary of the main kernel. Non-main interference regions are identified, and the interference intrusion status is determined based on the layer of the perturbation envelope into which the non-main interference regions fall. Based on the interference intrusion status, the main judgment key frames are selected, and the local texture features, semantic content features and effective global features of each main judgment key frame are extracted. The features are weighted and fused according to the interference intrusion status and the temporal position of the shot in the whole video to generate video-level candidate recall features. Based on video-level candidate recall features, candidate videos are recalled from the video library. A shot matching edge that maintains the sequence of shots is established between the video to be processed and the candidate videos. Shot closed chains are constructed for consecutively matched shots. The video deduplication results are output according to the coverage of the shot closed chain.

[0007] The present invention is further configured to select representative keyframes from each shot segment that meet preset conditions for sharpness and stability, including: Extract multiple candidate keyframes within each shot segment according to a preset frame interval; Calculate the sharpness value of each candidate keyframe and the area of ​​the stable overlapping region between each candidate keyframe and its preceding and following adjacent frames after position compensation. From the candidate keyframes whose sharpness values ​​reach the preset sharpness threshold, select the candidate keyframe with the largest stable overlapping area as the representative keyframe. When there are multiple candidate keyframes with the same stable overlapping area, the candidate keyframe with the smallest time distance from the midpoint of the shot is selected as the representative keyframe. For shot segments whose duration exceeds the preset duration, representative keyframes are selected for the beginning, middle, and end of the shot.

[0008] The present invention is further configured to generate a main body kernel based on content regions that maintain structural stability after position compensation in consecutive adjacent frames representing keyframes, including: Position compensation is performed on the keyframe and its preceding and following adjacent frames. Extract connected content regions in each frame after position compensation; Connected content regions that coexist in consecutive adjacent frames and whose positional offset does not exceed a preset displacement threshold are defined as stable overlapping regions. The stable overlapping region is used as the main core; When multiple stable overlapping regions exist, they are merged to form a composite core.

[0009] The present invention is further configured to form a perturbation envelope by extending outward in layers based on the boundary of the main core; identify non-main interference regions, and determine the interference intrusion state according to the layer of the perturbation envelope into which the non-main interference regions fall, including: Starting from the boundary of the main core, the inner perturbation envelope, the middle perturbation envelope, and the outer perturbation envelope are formed outward in sequence according to the preset expansion step size; Identify non-subjective interference regions in keyframes; Calculate the overlap area between the non-main interference region and the inner, middle, and outer disturbance envelopes, respectively. When the overlap area between the non-main interference region and the inner perturbation envelope reaches the preset inner intrusion threshold, the corresponding key frame is determined to be in the inner intrusion state. When the overlap area between the non-main interference region and the inner perturbation envelope does not reach the preset inner intrusion threshold, but the overlap area between the non-main interference region and the middle perturbation envelope reaches the preset middle intrusion threshold, the corresponding key frame is determined to be in the middle intrusion state. When the overlap area between the non-subject interference region and the inner and middle perturbation envelopes does not reach the corresponding intrusion threshold, and the overlap area with the outer perturbation envelope reaches the preset outer intrusion threshold, the corresponding representative keyframe is determined to be in an outer intrusion state.

[0010] The present invention is further configured to: filter keyframes for judgment based on the interference intrusion state, and extract local texture features, semantic content features, and effective global image features of each keyframe, including: The representative keyframe in the outer layer intrusion state is determined as the first main judgment keyframe. Local texture features and semantic content features are extracted in the main kernel region, and effective global features are extracted in the effective image region after removing non-main interference areas. The representative keyframe in the middle layer intrusion state is determined as the second main judgment keyframe, and local texture features and semantic content features are extracted in the main body kernel region; Representative keyframes in the inner layer intrusion state are identified as auxiliary keyframes, and local texture features are extracted in the main kernel region as auxiliary verification features.

[0011] The present invention is further configured to perform weighted fusion of features based on the interference / intrusion state and the temporal position of the shot in the entire video to generate video-level candidate recall features, including: When the main judgment keyframe is in an outer intrusion state, the preset first state weight is invoked. When the main judgment keyframe is in the middle layer intrusion state, the preset second state weight is invoked; When the main judgment keyframe is in the inner intrusion state, the preset third state weight is invoked; Calculate the normalized temporal position of the shot to which each key frame belongs in the entire video; The normalized temporal position is compared with the preset segmentation threshold, and the main judgment key frame is divided into the first segment key frame, the middle segment key frame and the last segment key frame, and the preset first temporal weight, second temporal weight and third temporal weight are called respectively. The local texture features, semantic content features, and effective global features of each key frame are multiplied by their corresponding state weights and temporal weights, and then accumulated in the order of the shots to generate video-level candidate recall features.

[0012] The present invention is further configured to recall candidate videos from a video library based on video-level candidate recall features, and establish a shot matching edge between the video to be processed and the candidate videos that maintains the sequence of shots, including: The similarity of the video-level candidate recall features of the video to be processed with the pre-stored video-level candidate recall features of each video in the video library is compared, and the video with the similarity reaching the preset recall threshold is selected as the candidate video. Retrieve the sequence of shot segments and representative keyframe features corresponding to the candidate video; When the feature similarity between the main keyframe in the video to be processed and the representative keyframe in the candidate video reaches a preset matching threshold, a shot matching edge is established between the corresponding shots. Remove matching edges where the order of shots in the video to be processed and the candidate video is reversed.

[0013] The present invention is further configured to construct a shot chain for consecutively matched shots, and output video deduplication results based on the coverage of the shot chain, including: A series of matched shots with the same sequence and the number of unmatched shots between adjacent matched edges not exceeding a preset chain break tolerance are defined as the same closed chain. Calculate the cumulative coverage time of each shot's closed loop in the video to be processed, and then calculate the proportion of the cumulative coverage time of each shot's closed loop to the total duration of the video to be processed. When the cumulative coverage time reaches the preset duplicate judgment threshold, and the length of the first and last uncovered segments of the video to be processed does not exceed the preset first and last length thresholds, the duplicate video result is output. When the cumulative coverage time reaches the preset duplicate judgment threshold, and the length of the first segment of the video to be processed that is not covered or the length of the last segment that is not covered exceeds the preset first and last length threshold, the result of the video with the same source at the beginning and end is output. When the cumulative coverage time percentage does not reach the preset duplicate judgment threshold but reaches the preset local overlap threshold, the local overlap video result is output. When the cumulative coverage time percentage does not reach the preset local overlap threshold, output non-same-source video results.

[0014] This invention also provides a video deduplication system based on keyframe weighted feature fusion, for implementing the above-mentioned video deduplication method based on keyframe weighted feature fusion, comprising: Storyboard frame selection module: acquires the video to be processed and performs shot segmentation, and selects representative keyframes from each shot segment that meet preset conditions for clarity and stability; State determination module: Generates a main kernel based on the content region of consecutive adjacent frames in the representative keyframe that maintains structural stability after position compensation, and expands outward in layers based on the boundary of the main kernel to form a perturbation envelope; identifies non-main interference regions, and determines the interference intrusion state based on the layer of the perturbation envelope into which the non-main interference regions fall; Feature fusion module: Select key frames for judgment based on interference intrusion status, extract local texture features, semantic content features and effective global features of each key frame, and perform weighted fusion of features according to interference intrusion status and the temporal position of the shot in the whole video to generate video-level candidate recall features. Deduplication output module: Based on video-level candidate recall features, candidate videos are recalled from the video library. A shot matching edge that maintains the sequential order of shots is established between the video to be processed and the candidate videos. Shot closed chains are constructed for consecutively matched shots. The deduplication results of the video are output according to the coverage of the shot closed chain.

[0015] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer processor, causes the computer to perform the video deduplication method based on keyframe weighted feature fusion as described in any of the preceding claims.

[0016] This invention provides a video deduplication method, system, and medium based on keyframe weighted feature fusion. The method involves acquiring the video to be processed and segmenting it into shots. Representative keyframes with satisfactory clarity and stability are selected from each shot segment. A main body kernel is generated based on the content region of consecutive adjacent frames within the representative keyframes, maintaining structural stability after position compensation. A perturbation envelope is formed by expanding outwards from the boundary of the main body kernel. Non-main interference regions are identified, and the interference intrusion state is determined based on the layer of the perturbation envelope into which the non-main interference regions fall. Keyframes are selected based on the interference intrusion state, and local texture features, semantic content features, and effective global features of each keyframe are extracted. These features are weighted and fused according to the interference intrusion state and the temporal position of the shot in the entire video to generate video-level candidate recall features. Candidate videos are recalled from a video library based on these video-level candidate recall features. A shot matching edge maintaining the shot sequence is established between the video to be processed and the candidate videos. A shot closed chain is constructed for consecutively matched shots. The video deduplication result is output based on the coverage of the shot closed chain. The beneficial effects include: 1. By generating a main kernel based on representative keyframes and expanding outwards in layers based on the boundary of the main kernel to form a perturbation envelope, and then determining the interference intrusion state based on the layer where the non-main interference area falls into the perturbation envelope, the main content of the video can be distinguished from peripheral interference content such as subtitles, watermarks, logos, borders and patches. This can reduce the impact of non-main interference content on the deduplication results in the feature extraction and weighted fusion stages, making the generated video-level candidate recall features more representative of the main content of the video. This is beneficial to improving the accuracy of identifying duplicate and near-duplicate videos and reducing misjudgments caused by platform packaging, additional information on the screen or partial occlusion. 2. By extracting and fusing features from keyframes, a shot matching edge is established between candidate videos to maintain the sequential order of shots. A shot chain is constructed for consecutively matched shots. The video deduplication result is then output based on the coverage of the shot chain. This can effectively distinguish between complete duplicate videos, videos with modified beginnings and ends that are from the same source, partially overlapping videos, and videos that are not from the same source. 3. By utilizing video-level candidate recall features to recall candidate videos from the video library, and then performing shot-level closed-chain verification on the candidate videos, we can avoid performing shot-by-shot or frame-by-frame fine comparison of the entire video library. This balances recall efficiency and judgment accuracy, and can not only meet the rapid screening needs in large-scale video library scenarios, but also improve the final judgment accuracy through closed-chain verification after narrowing down the candidate range.

[0017] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 The flowchart illustrates a video deduplication method based on keyframe weighted feature fusion, which is an exemplary embodiment of the present invention. Detailed Implementation

[0019] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0020] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0021] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0022] Example 1: Video deduplication methods based on keyframe weighted feature fusion, such as Figure 1 As shown, it includes: The video to be processed is acquired and segmented into shots. Representative keyframes that meet the preset conditions for clarity and stability are selected from each shot segment. The main kernel is generated based on the content regions that maintain structural stability after position compensation in consecutive adjacent frames in the representative keyframe. The perturbation envelope is formed by expanding outward in layers based on the boundary of the main kernel. Non-main interference regions are identified, and the interference intrusion status is determined based on the layer of the perturbation envelope into which the non-main interference regions fall. Based on the interference intrusion status, the main judgment key frames are selected, and the local texture features, semantic content features and effective global features of each main judgment key frame are extracted. The features are weighted and fused according to the interference intrusion status and the temporal position of the shot in the whole video to generate video-level candidate recall features. Based on video-level candidate recall features, candidate videos are recalled from the video library. A shot matching edge that maintains the sequence of shots is established between the video to be processed and the candidate videos. Shot closed chains are constructed for consecutively matched shots. The video deduplication results are output according to the coverage of the shot closed chain.

[0023] Specifically, the method can be deployed in a video content processing server or a distributed video processing platform. The processing server includes at least a processor, a memory, and a database for storing the video to be processed, candidate video indexes, and intermediate results.

[0024] After the video is input, preprocessing is performed first. Preprocessing includes decoding to obtain the original frame sequence, unifying the frame rate, extracting basic attributes such as video duration, resolution, and aspect ratio, and converting the video into a standard frame sequence suitable for content analysis. In one specific embodiment, the video is uniformly processed into a sequence of 25 to 30 frames per second. While preserving the original resolution, a thumbnail analysis sequence is generated. The thumbnail analysis sequence is used for shot segmentation and initial selection of candidate keyframes, while the original resolution sequence is used for subject kernel extraction and high-precision feature calculation. For videos with long black screens, solid color pages, intro logo pages, or static cover pages at the beginning and end, these parts can be marked as additional segments at the beginning and end by detecting the average frame brightness and inter-frame difference, so as to facilitate subsequent determination of the length of the beginning and end segments.

[0025] Shot segmentation is accomplished using a combined change analysis of adjacent frames. The processor calculates the color distribution difference, edge change degree, and brightness change degree for adjacent frames in the thumbnail analysis sequence, and determines shot boundaries by combining motion continuity. For example, for any two adjacent frames, the color histogram difference is calculated first, then the change ratio of edge pixel sets is calculated, and brightness abrupt changes are judged simultaneously. When both color and edge changes exceed the segmentation threshold, and the preceding and following frames do not satisfy a continuous translation or scaling motion relationship, that position is recorded as the shot boundary. For fade-in / fade-out or dissolve transitions, a sliding window of 3 to 5 consecutive frames can be used to compare cumulative differences to avoid mistakenly classifying transition frames into adjacent shots. After shot segmentation, multiple shot segments are obtained in chronological order, each shot segment having a start frame number, end frame number, and segment duration.

[0026] When selecting representative keyframes within each shot segment, instead of directly extracting frames at fixed time points, candidate keyframes are first extracted at preset frame intervals. In one specific embodiment, the processor extracts multiple candidate keyframes from the shot segment at intervals of 8 to 12 frames. For shorter shot segments, at least one candidate keyframe is extracted. For longer shot segments, the segment is divided into a beginning, middle, and end segment, and candidate keyframes are extracted from each segment. Subsequently, the sharpness value of each candidate keyframe and the area of ​​the stable overlap region between the candidate keyframe and its preceding and following adjacent frames after position compensation are calculated. The sharpness value can be obtained based on the high-frequency information content, edge energy, or Laplacian response intensity of the image; the area of ​​the stable overlap region reflects the structural persistence of the candidate keyframe in adjacent frames. The processor first retains candidate keyframes whose sharpness values ​​reach a preset sharpness threshold, and then selects the candidate keyframe with the largest stable overlap region area as the representative keyframe. When multiple candidate keyframes have the same stable overlap region area, the candidate keyframe with the smallest time distance from the midpoint of the shot segment is selected as the representative keyframe. This method can avoid directly extracting blurry frames, flickering frames, or structurally unstable frames.

[0027] After the representative keyframe is determined, the processor generates a subject kernel based on the representative keyframe and its preceding and following frames. Position compensation is used to eliminate the influence of slight camera shake, image translation, or scaling on the region overlap determination. In one specific embodiment, the processor estimates the inter-frame displacement relationship for the representative keyframe and its preceding and following frames based on feature point matching or block matching methods, and aligns the adjacent frames to the coordinate system of the representative keyframe. Subsequently, connected content regions are extracted in each frame after position compensation. These connected content regions can be obtained based on salient region segmentation, target region detection, or continuous texture region extraction. Connected content regions that coexist in consecutive adjacent frames and whose positional offset does not exceed a preset displacement threshold are determined as stable overlapping regions, and these stable overlapping regions are used as subject kernels. When multiple stable overlapping regions exist, they are merged to form a composite subject kernel. For example, in a product advertising video, the product subject area and the person's hand operation area may simultaneously meet the stable overlap condition. In this case, the two are merged to form a composite subject kernel to improve the integrity of the subject content coverage.

[0028] After the main kernel is generated, a perturbation envelope is formed by expanding outwards in layers from the main kernel boundary. The perturbation envelope is not a regular circle, but rather an envelope region obtained by expanding outwards according to the shape of the main kernel boundary. In one specific implementation, starting from the main kernel boundary, an inner perturbation envelope, a middle perturbation envelope, and an outer perturbation envelope are formed sequentially according to a preset expansion step size. The expansion step size can be set to 1% to 3% of the image's shorter side length. Subsequently, non-subject interference regions in keyframes are identified. Non-subject interference regions include at least one of the following: subtitle regions, watermark regions, logo regions, border regions, and patch occlusion regions. Subtitle regions can be obtained through high-contrast text region detection at fixed positions; watermark and logo regions can be obtained through persistent small corner regions or transparent overlay regions; border regions can be obtained through continuous black or colored border detection; and patch occlusion regions can be obtained through rigid overlay regions unrelated to the original video subject's motion.

[0029] The processor calculates the overlap area between the non-main interference region and the inner, middle, and outer perturbation envelopes, respectively, and determines the interference intrusion state accordingly. Specifically, when the overlap area between the non-main interference region and the inner perturbation envelope reaches a preset inner intrusion threshold, the corresponding keyframe is determined to be in an inner intrusion state; when the overlap area between the non-main interference region and the inner perturbation envelope does not reach the preset inner intrusion threshold, but the overlap area with the middle perturbation envelope reaches a preset middle intrusion threshold, the corresponding keyframe is determined to be in a middle intrusion state; when the overlap area between the non-main interference region and both the inner and middle perturbation envelopes does not reach the corresponding intrusion threshold, but the overlap area with the outer perturbation envelope reaches a preset outer intrusion threshold, the corresponding keyframe is determined to be in an outer intrusion state. The inner layer intrusion state indicates that the interfering content has approached or intruded into the edge of the main body core, and has a significant impact on the expression of the main content; the middle layer intrusion state indicates that the interfering content is close to the main body core but has not penetrated into the core of the main body; the outer layer intrusion state indicates that the interfering content is mainly distributed on the periphery of the main body, and has a relatively small impact on the main content.

[0030] Keyframes are selected for primary judgment based on their intrusion state, and features are extracted. In one specific embodiment, representative keyframes in the outer intrusion state are designated as the first primary judgment keyframes. Local texture features and semantic content features are extracted from these keyframes within the main body kernel region, and effective global image features are extracted from the effective image region after removing non-main body interference regions. Representative keyframes in the middle intrusion state are designated as the second primary judgment keyframes. Local texture features and semantic content features are extracted from these keyframes within the main body kernel region, while effective global image features are not extracted or are weakened. Representative keyframes in the inner intrusion state are designated as auxiliary keyframes, and only local texture features are extracted from the main body kernel region as auxiliary verification features. Local texture features can be keypoint texture description, local block texture encoding, or local convolution features; semantic content features can be feature vectors obtained by high-level semantic encoding of the main body kernel region; effective global image features can be overall layout feature vectors after removing interference. This branching feature extraction strategy enables keyframes in different intrusion states to participate in subsequent calculations in different ways, avoiding excessive impact of strongly interfering keyframes on the overall recall results.

[0031] After feature extraction, the processor weights and fuses the features according to the intrusion state and the temporal position of the shot in the entire video to generate video-level candidate recall features. For state weights, the outer intrusion state corresponds to the first state weight, the middle intrusion state corresponds to the second state weight, and the inner intrusion state corresponds to the third state weight. The first, second, and third state weights are pre-set different weights, such as 1, 0.7, and 0.3 respectively, or learned offline from training samples. For temporal weights, the normalized temporal position of the shot to which each keyframe belongs in the entire video is first calculated. Then, the normalized temporal position is compared with a preset segmentation threshold to divide the keyframe into first, middle, and last keyframes, and the preset first, second, and third temporal weights are applied respectively. Considering that the beginning and end of the video are more easily packaged by the platform, have their intros and outros replaced, or have QR codes inserted, the temporal weight of the middle keyframe is usually set higher than that of the first and last keyframes, for example, 1, 0.8, and 0.8. The processor multiplies the local texture features, semantic content features, and effective global features of each keyframe by their corresponding state weights and temporal weights, and then sums or weights them according to the order of the shots to obtain video-level candidate recall features. If there are multiple representative keyframes in a shot segment, shot-level features are first formed within the same shot segment, and then video-level features are formed according to the order of the shots.

[0032] After generating video-level candidate recall features, the processor recalls candidate videos from the video library. The video library pre-stores video-level candidate recall features, shot sequence sequences, and representative keyframe features for each historical video. The processor compares the similarity of the video-level candidate recall features of the video to be processed with the pre-stored video-level candidate recall features of each video in the video library, using cosine similarity, inverse Euclidean distance, or normalized correlation as comparison metrics. When the similarity reaches a preset recall threshold, the corresponding video is selected as a candidate video. To balance recall rate and computational efficiency, in one specific embodiment, the top-ranking videos in terms of similarity can be selected first as candidate videos, and then shot-level fine-tuning can be performed on these candidate videos.

[0033] When establishing shot matching edges that maintain the sequential order of shots between the video to be processed and the candidate videos, the processor retrieves the shot segment sequence and representative keyframe features corresponding to the candidate videos, and compares the features of the main judgment keyframes in the video to be processed with those of the representative keyframes in the candidate videos. When the comprehensive similarity of the local texture features, semantic content features, and effective global features of the two reaches a preset matching threshold, a shot matching edge is established between the corresponding shots. The comprehensive similarity can be obtained by fusing the similarity of each feature branch according to a preset ratio, or by prioritizing the comparison of the same branch when the main judgment keyframe types are the same. After establishing the shot matching edges, shot matching edges with reversed shot order in the video to be processed and the candidate videos are removed, and only shot matching edges with consistent order are retained, thereby ensuring that the subsequent shot closure chain can reflect the true content continuity.

[0034] The processor constructs shot chains for consecutively matched shots. Shot chains with consistent shot order and a number of unmatched shots between adjacent matched edges not exceeding a preset chain break tolerance are defined as the same shot chain. For example, the chain break tolerance can be set to one or two shots, indicating that a small number of transition shots are allowed to be unmatched when forming a chain, but long-distance jump matching is not allowed. The processor then calculates the cumulative coverage time of each shot chain in the video to be processed, and calculates the proportion of the cumulative coverage time to the total video duration. If the cumulative coverage time proportion reaches a preset duplication threshold, and the lengths of the uncovered sections at the beginning and end of the video do not exceed preset beginning-end length thresholds, a duplicate video result is output. If the cumulative coverage time proportion reaches the preset duplication threshold, but the length of the uncovered section at the beginning or end exceeds the preset beginning-end length threshold, a video result indicating the beginning and end are from the same source is output. If the cumulative coverage time proportion does not reach the preset duplication threshold but reaches a preset local overlap threshold, a locally overlapped video result is output. If the cumulative coverage time proportion does not reach the preset local overlap threshold, a non-same-source video result is output. In one specific embodiment, the repetition determination threshold can be set to 0.8 to 0.9, the local overlap threshold can be set to 0.3 to 0.5, and the beginning and end length threshold can be set to 5% to 10% of the total video duration or a fixed number of seconds threshold.

[0035] Furthermore, in a feasible specific application example, for Video A and Video B, both are different platform versions of the same product advertisement. Video A is 30 seconds long, with the first two seconds being the platform intro, the middle 26 seconds being the product display content, and the last two seconds being the QR code page; Video B is 28 seconds long, without a platform intro, but with an additional one-second brand logo page at the end. After the processor performs shot segmentation on both videos, it obtains several shot fragments. Within each shot fragment, candidate keyframes are extracted at preset frame intervals, and representative keyframes are selected; position compensation is performed on the representative keyframe and its adjacent frames to generate the main body kernel; non-subject interference areas are identified through subtitle detection and watermark detection, and then combined with a three-layer perturbation envelope to determine that most keyframes are in an outer layer intrusion state or a middle layer intrusion state. Subsequently, local texture features, semantic content features, and effective global image features are extracted and weighted and fused according to corresponding weights to obtain video-level candidate recall features. After the processor recalls similar videos from the video library, it establishes a shot matching edge between Video A and Video B, constructing a shot closed chain covering the middle main content. Since the cumulative coverage time of the closed-loop shot reaches the duplicate judgment threshold, but the first segment of video A has an extra intro and the last segment of video B has an extra logo page, the length of the uncovered section in the first or last segment exceeds the preset first and last length threshold. Therefore, the output is a video with the same source at the beginning and end.

[0036] For example, Video C is a montage video that borrows only three short shots from the original advertisement and inserts a large amount of other content. Although some shots in Video C have high feature similarity to Video A, the number of unmatched shots between consecutive matched shots exceeds the preset chain break tolerance, making it impossible to form a closed shot chain covering the main content. The cumulative coverage time does not reach the local overlap threshold. Therefore, it outputs a non-originating video result or a partially overlapping video result, rather than being mistakenly judged as a complete duplicate video. This example shows that the method can effectively distinguish between videos with modified beginnings and ends from the same source, partially borrowed videos, and non-originating videos.

[0037] In specific implementations, the candidate keyframe extraction interval, sharpness threshold, displacement threshold, intrusion threshold, state weight, temporal weight, recall threshold, matching threshold, link break tolerance, and various judgment thresholds can all be set or adaptively updated based on the actual video type, video resolution, business scenario, and sample statistical results. Different parameter combinations can be used for e-commerce videos, advertising videos, short video content, and media asset archives, but the processing logic remains consistent: video deduplication is achieved by separating the main subject from non-subject interference regions, combining keyframe weighted feature fusion, and shot link verification. Therefore, this method is applicable not only to single-machine deployments but also to batch video deduplication in large-scale video library environments.

[0038] Example 2: This exemplary video deduplication system based on keyframe weighted feature fusion is used to implement the aforementioned video deduplication method based on keyframe weighted feature fusion, including: Storyboard frame selection module: acquires the video to be processed and performs shot segmentation, and selects representative keyframes from each shot segment that meet preset conditions for clarity and stability; State determination module: Generates a main kernel based on the content region of consecutive adjacent frames in the representative keyframe that maintains structural stability after position compensation, and expands outward in layers based on the boundary of the main kernel to form a perturbation envelope; identifies non-main interference regions, and determines the interference intrusion state based on the layer of the perturbation envelope into which the non-main interference regions fall; Feature fusion module: Select key frames for judgment based on interference intrusion status, extract local texture features, semantic content features and effective global features of each key frame, and perform weighted fusion of features according to interference intrusion status and the temporal position of the shot in the whole video to generate video-level candidate recall features. Deduplication output module: Based on video-level candidate recall features, candidate videos are recalled from the video library. A shot matching edge that maintains the sequential order of shots is established between the video to be processed and the candidate videos. Shot closed chains are constructed for consecutively matched shots. The deduplication results of the video are output according to the coverage of the shot closed chain.

[0039] It should be noted that the video deduplication system based on keyframe weighted feature fusion provided in the above embodiments and the video deduplication method based on keyframe weighted feature fusion provided in the above embodiments belong to the same concept. The specific methods of execution of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the video deduplication system based on keyframe weighted feature fusion provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0040] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the video deduplication method based on keyframe weighted feature fusion as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0041] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A video deduplication method based on keyframe weighted feature fusion, characterized in that, include: The video to be processed is acquired and segmented into shots. Representative keyframes that meet the preset conditions for clarity and stability are selected from each shot segment. The main kernel is generated based on the content regions that maintain structural stability after position compensation in consecutive adjacent frames of the representative keyframe. The perturbation envelope is formed by expanding outward in layers based on the boundary of the main kernel. Identify non-primary interference regions and determine the interference intrusion status based on the layer of the disturbance envelope where the non-primary interference regions fall; Based on the interference intrusion status, the main judgment key frames are selected, and the local texture features, semantic content features and effective global features of each main judgment key frame are extracted. The features are weighted and fused according to the interference intrusion status and the temporal position of the shot in the whole video to generate video-level candidate recall features. Based on video-level candidate recall features, candidate videos are recalled from the video library. A shot matching edge that maintains the sequence of shots is established between the video to be processed and the candidate videos. Shot closed chains are constructed for consecutively matched shots. The video deduplication results are output according to the coverage of the shot closed chain.

2. The video deduplication method based on keyframe weighted feature fusion according to claim 1, characterized in that, Select representative keyframes from each shot segment that meet preset conditions for sharpness and stability, including: Extract multiple candidate keyframes within each shot segment according to a preset frame interval; Calculate the sharpness value of each candidate keyframe and the area of ​​the stable overlapping region between each candidate keyframe and its preceding and following adjacent frames after position compensation. From the candidate keyframes whose sharpness values ​​reach the preset sharpness threshold, select the candidate keyframe with the largest stable overlapping area as the representative keyframe. When there are multiple candidate keyframes with the same stable overlapping area, the candidate keyframe with the smallest time distance from the midpoint of the shot is selected as the representative keyframe. For shot segments whose duration exceeds the preset duration, representative keyframes are selected for the beginning, middle, and end of the shot.

3. The video deduplication method based on keyframe weighted feature fusion according to claim 1, characterized in that, The main body kernel is generated based on the content regions in the representative keyframes that maintain structural stability after position compensation, including: Position compensation is performed on the keyframe and its preceding and following adjacent frames. Extract connected content regions in each frame after position compensation; Connected content regions that coexist in consecutive adjacent frames and whose positional offset does not exceed a preset displacement threshold are defined as stable overlapping regions. The stable overlapping region is used as the main core; When multiple stable overlapping regions exist, they are merged to form a composite core.

4. The video deduplication method based on keyframe weighted feature fusion according to claim 1, characterized in that, The perturbation envelope is formed by extending outward in layers from the boundary of the main core. Identify non-primary interference regions and determine the interference intrusion state based on the layer of the perturbation envelope into which the non-primary interference regions fall, including: Starting from the boundary of the main core, the inner perturbation envelope, the middle perturbation envelope, and the outer perturbation envelope are formed outward in sequence according to the preset expansion step size; Identify non-subjective interference regions in keyframes; Calculate the overlap area between the non-main interference region and the inner, middle, and outer disturbance envelopes, respectively. When the overlap area between the non-main interference region and the inner perturbation envelope reaches the preset inner intrusion threshold, the corresponding key frame is determined to be in the inner intrusion state. When the overlap area between the non-main interference region and the inner perturbation envelope does not reach the preset inner intrusion threshold, but the overlap area between the non-main interference region and the middle perturbation envelope reaches the preset middle intrusion threshold, the corresponding key frame is determined to be in the middle intrusion state. When the overlap area between the non-subject interference region and the inner and middle perturbation envelopes does not reach the corresponding intrusion threshold, and the overlap area with the outer perturbation envelope reaches the preset outer intrusion threshold, the corresponding representative keyframe is determined to be in an outer intrusion state.

5. The video deduplication method based on keyframe weighted feature fusion according to claim 4, characterized in that, Based on the interference intrusion status, keyframes for judgment are selected, and local texture features, semantic content features, and effective global image features of each keyframe are extracted, including: The representative keyframe in the outer layer intrusion state is determined as the first main judgment keyframe. Local texture features and semantic content features are extracted in the main kernel region, and effective global features are extracted in the effective image region after removing non-main interference areas. The representative keyframe in the middle layer intrusion state is determined as the second main judgment keyframe, and local texture features and semantic content features are extracted in the main body kernel region; Representative keyframes in the inner layer intrusion state are identified as auxiliary keyframes, and local texture features are extracted in the main kernel region as auxiliary verification features.

6. The video deduplication method based on keyframe weighted feature fusion according to claim 4, characterized in that, Features are weighted and fused based on the interference / intrusion status and the temporal position of the shot within the entire video to generate video-level candidate recall features, including: When the main judgment keyframe is in an outer intrusion state, the preset first state weight is invoked. When the main judgment keyframe is in the middle layer intrusion state, the preset second state weight is invoked; When the main judgment keyframe is in the inner intrusion state, the preset third state weight is invoked; Calculate the normalized temporal position of the shot to which each key frame belongs in the entire video; The normalized temporal position is compared with the preset segmentation threshold, and the main judgment key frame is divided into the first segment key frame, the middle segment key frame and the last segment key frame, and the preset first temporal weight, second temporal weight and third temporal weight are called respectively. The local texture features, semantic content features, and effective global features of each key frame are multiplied by their corresponding state weights and temporal weights, and then accumulated in the order of the shots to generate video-level candidate recall features.

7. The video deduplication method based on keyframe weighted feature fusion according to claim 1, characterized in that, Based on video-level candidate recall features, candidate videos are recalled from the video library. A shot-matching edge is established between the video to be processed and the candidate videos, maintaining the sequential order of shots. This includes: The similarity of the video-level candidate recall features of the video to be processed with the pre-stored video-level candidate recall features of each video in the video library is compared, and the video with the similarity reaching the preset recall threshold is selected as the candidate video. Retrieve the sequence of shot segments and representative keyframe features corresponding to the candidate video; When the feature similarity between the main keyframe in the video to be processed and the representative keyframe in the candidate video reaches a preset matching threshold, a shot matching edge is established between the corresponding shots. Remove matching edges where the order of shots in the video to be processed and the candidate video is reversed.

8. The video deduplication method based on keyframe weighted feature fusion according to claim 1, characterized in that, Construct a shot chain for consecutively matched shots, and output the video deduplication results based on the coverage of the shot chain, including: A series of matched shots with the same sequence and the number of unmatched shots between adjacent matched edges not exceeding a preset chain break tolerance are defined as the same closed chain. Calculate the cumulative coverage time of each shot's closed loop in the video to be processed, and then calculate the proportion of the cumulative coverage time of each shot's closed loop to the total duration of the video to be processed. When the cumulative coverage time reaches the preset duplicate judgment threshold, and the length of the first and last uncovered segments of the video to be processed does not exceed the preset first and last length thresholds, the duplicate video result is output. When the cumulative coverage time reaches the preset duplicate judgment threshold, and the length of the first segment of the video to be processed that is not covered or the length of the last segment that is not covered exceeds the preset first and last length threshold, the result of the video with the same source at the beginning and end is output. When the cumulative coverage time percentage does not reach the preset duplicate judgment threshold but reaches the preset local overlap threshold, the local overlap video result is output. When the cumulative coverage time percentage does not reach the preset local overlap threshold, output non-same-source video results.

9. A video deduplication system based on keyframe weighted feature fusion, used to implement the video deduplication method based on keyframe weighted feature fusion as described in any one of claims 1-8, characterized in that, include: Storyboard frame selection module: acquires the video to be processed and performs shot segmentation, and selects representative keyframes from each shot segment that meet preset conditions for clarity and stability; State determination module: Generates a main kernel based on the content region of consecutive adjacent frames in the representative keyframe that maintains structural stability after position compensation, and expands outward in layers based on the boundary of the main kernel to form a perturbation envelope; identifies non-main interference regions, and determines the interference intrusion state based on the layer of the perturbation envelope into which the non-main interference regions fall; Feature fusion module: Select key frames for judgment based on interference intrusion status, extract local texture features, semantic content features and effective global features of each key frame, and perform weighted fusion of features according to interference intrusion status and the temporal position of the shot in the whole video to generate video-level candidate recall features. Deduplication output module: Based on video-level candidate recall features, candidate videos are recalled from the video library. A shot matching edge that maintains the sequential order of shots is established between the video to be processed and the candidate videos. Shot closed chains are constructed for consecutively matched shots. The deduplication results of the video are output according to the coverage of the shot closed chain.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the video deduplication method based on keyframe weighted feature fusion as described in any one of claims 1 to 8.

Images