Video time positioning method and device, and electronic device

Abstract

The present disclosure relates to a video time positioning method and device, and an electronic device. The time positioning method comprises detecting the authenticity of a to-be-detected video, obtaining detection results for each video frame in the to-be-detected video, and the detection results representing the probability of forgery of the video frame. The probability of forgery of each video frame is input into a time positioning model for calculation to obtain time information of each forged segment of the to-be-detected video and the probability of forgery of each forged segment. The present disclosure can accurately obtain the position and probability of forgery of the forged segment in the to-be-detected video without manually setting a sliding window to divide the to-be-detected video into multiple segments.

Description

Technical Field

[0001] This disclosure relates to the field of deepfake detection technology, and in particular to a video time positioning method and apparatus, and electronic equipment. Background Technology

[0002] With the development of deepfake generation technology, the work of deepfake detection has also gradually emerged. The targets of detection have evolved from the poor-quality videos generated in the early days of deepfakes to the forged videos that are now difficult to distinguish with the naked eye. The quality of the forged video frames is not only getting higher and higher, but the consistency between the original and the forged versions is also getting better and better.

[0003] The task of forgery time localization is to locate the start and end times of forged segments in videos that may contain localized forgeries. Current research on this task is limited. A common approach is to divide the video into multiple overlapping segments according to a certain window size, and then classify these segments separately using existing deepfake detection methods. This approach not only has low accuracy but also requires manual design of parameters such as window size. Therefore, this area has not been fully explored, and there is an urgent need to develop new methods to improve localization performance. Summary of the Invention

[0004] In view of this, this disclosure proposes a video time localization method, apparatus, and electronic device that can accurately obtain the location of the forged segment and the probability of forgery in the video without requiring the user to manually set a sliding window to divide the video to be detected into multiple segments.

[0005] According to one aspect of this disclosure, a video temporal localization method is provided, the method comprising: detecting the authenticity of a video to be detected, obtaining detection results for each video frame in the video to be detected, the detection results representing the forgery probability of the video frame; inputting the forgery probability of each video frame into a temporal localization model for calculation, to obtain temporal information of each forged segment of the video to be detected and the forgery probability of each forged segment.

[0006] In one possible implementation, detecting the authenticity of the video to be detected and obtaining detection results for each video frame in the video to be detected includes: extracting each target frame pair and a first audio feature for each target frame pair from the video to be detected, wherein each target frame pair includes two adjacent video frames, and the first audio feature includes Mel-spectrum information; inputting each target frame pair and the first audio feature for each target frame pair into a forgery detection model for calculation to obtain a score for each target frame pair, wherein the score represents the forgery probability of the target frame pair; and determining the detection result for each video frame in the video to be detected based on the score of each target frame pair.

[0007] In one possible implementation, the time localization model includes: a feature extraction module, used to perform multiple feature extractions and multiple upsamplings based on the forgery probability of each video frame to obtain multiple target extracted features; an attention layer, used to perform feature fusion based on the multiple target extracted features to obtain fused features; and a combination module, used to determine the time information of each forged segment and the forgery probability of each forged segment based on the fused features.

[0008] In one possible implementation, the feature extraction module includes: a plurality of sequentially connected first-type multi-layer convolutional modules, each of which is used to extract features from the forgery probability of each video frame, or to extract features from the first-layer convolutional features extracted from the connected previous first-type multi-layer convolutional module, to obtain first-layer convolutional features; at least one convolutional module, each of which is used to perform a one-dimensional convolution operation on the first-layer convolutional features extracted from the connected first-type multi-layer convolutional modules, to obtain convolutional features; and at least one second-type multi-layer convolutional module, each of which is used to extract features from the first-layer convolutional features extracted from the connected first-type multi-layer convolutional modules. The system performs feature extraction, or extracts features from upsampled features from connected upsampling modules, to obtain second multi-layer convolutional extracted features; at least one feature merger is used to perform addition operations on the second multi-layer convolutional extracted features from connected second-type multi-layer convolutional modules and the convolutional features from connected convolutional modules to obtain merged features; at least one upsampling module is used to upsample the merged features from connected feature mergers to obtain upsampled features; wherein, the plurality of target extracted features include the second multi-layer convolutional extracted features output by the second multi-layer convolutional module connected to the first multi-layer convolutional module and the upsampled features output by each of the upsampling modules.

[0009] In one possible implementation, the plurality of sequentially connected first-type multi-layer convolutional modules include a first multi-layer convolutional module, a second multi-layer convolutional module, a third multi-layer convolutional module, and a fourth multi-layer convolutional module; the at least one convolutional module includes a first convolutional module, a second convolutional module, and a third convolutional module; the at least one second-type multi-layer convolutional module includes a fifth multi-layer convolutional module, a sixth multi-layer convolutional module, and a seventh multi-layer convolutional module; the at least one feature merger includes a first feature merger, a second feature merger, and a third feature merger; and the at least one upsampling module includes a first upsampling module, a second upsampling module, and a third upsampling module. A convolutional module is used to extract features from the forgery probability of each video frame to obtain a first extracted feature; the first convolutional module is used to perform a one-dimensional convolution operation on the first extracted feature to obtain a first convolutional feature; a second multi-layer convolutional module is used to extract features from the first extracted feature to obtain a second extracted feature; the second convolutional module is used to perform a one-dimensional convolution operation on the second extracted feature to obtain a second convolutional feature; a third multi-layer convolutional module is used to extract features from the second extracted feature to obtain a third extracted feature; the third convolutional module is used to perform a one-dimensional convolution operation on the third extracted feature to obtain a third convolutional feature. The fourth multi-layer convolutional module is used to extract features from the third extracted features to obtain a fourth extracted feature; the fifth multi-layer convolutional module is used to extract features from the fourth extracted features to obtain a fifth extracted feature; the first feature merger is used to perform an addition operation on the fifth extracted feature and the third convolutional feature to obtain a first merged feature; the first upsampling module is used to upsample the first merged feature to obtain a first upsampled feature; the sixth multi-layer convolutional module is used to extract features from the first upsampled feature to obtain a sixth extracted feature; the second feature merger is used to combine the sixth extracted feature and the second convolutional feature... The first feature extraction module performs an addition operation on the second merged feature to obtain a second merged feature; the second upsampling module upsamples the second merged feature to obtain a second upsampled feature; the seventh multi-layer convolution module extracts features from the second upsampled feature to obtain a seventh extracted feature; the third feature merger performs an addition operation on the seventh extracted feature and the first convolutional feature to obtain a third merged feature; the third upsampling module upsamples the third merged feature to obtain a third upsampled feature; wherein, the plurality of target extracted features include the fifth extracted feature, the first upsampled feature, the second upsampled feature, and the third upsampled feature.

[0010] In one possible implementation, the combination module includes: an eighth multi-layer convolutional module for extracting features based on the fused features to obtain an eighth extracted feature; a fourth convolutional module for performing a one-dimensional convolution operation on the eighth extracted feature to obtain the time information of each forged segment in the video to be detected, wherein the parameters of the fourth convolutional module are set for the time information output scenario; and a fifth convolutional module for performing a one-dimensional convolution operation on the eighth extracted feature to obtain the forgery probability of each forged segment in the video to be detected, wherein the parameters of the fifth convolutional module are set for the forgery probability output scenario.

[0011] In one possible implementation, the method further includes: acquiring a training sample set, the training sample set including multiple samples, each sample including a sample video and a sample video tag corresponding to the sample video, the sample video tag being used to indicate the authenticity of the corresponding sample video; training an initial model using the training sample set until the loss function value of the model meets a preset condition to obtain a time-localization model, the loss function used for training including a one-dimensional distance intersection-union function.

[0012] In one possible implementation, the method further includes: if it is determined that the time information of multiple forged segments overlaps, then the time information of the multiple forged segments is reduced and / or merged based on the forgery probability of each forged segment.

[0013] In one possible implementation, the method further includes: selecting, from all forged segments, forged segments with a forged probability greater than a preset threshold as the target result for the video to be detected.

[0014] According to another aspect of this disclosure, a video time positioning device is provided, the device comprising: a detection module, configured to detect the authenticity of a video to be detected and obtain detection results for each video frame in the video to be detected, the detection results representing the forgery probability of the video frame; and a calculation module, configured to input the forgery probability of each video frame into a time positioning model for calculation, and obtain time information of each forged segment of the video to be detected and the forgery probability of each forged segment.

[0015] In one possible implementation, detecting the authenticity of the video to be detected and obtaining detection results for each video frame in the video to be detected includes: extracting each target frame pair and a first audio feature for each target frame pair from the video to be detected, wherein each target frame pair includes two adjacent video frames, and the first audio feature includes Mel-spectrum information; inputting each target frame pair and the first audio feature for each target frame pair into a forgery detection model for calculation to obtain a score for each target frame pair, wherein the score represents the forgery probability of the target frame pair; and determining the detection result for each video frame in the video to be detected based on the score of each target frame pair.

[0016] In one possible implementation, the time localization model includes: a feature extraction module, used to perform multiple feature extractions and multiple upsamplings based on the forgery probability of each video frame to obtain multiple target extracted features; an attention layer, used to perform feature fusion based on the multiple target extracted features to obtain fused features; and a combination module, used to determine the time information of each forged segment and the forgery probability of each forged segment based on the fused features.

[0017] In one possible implementation, the feature extraction module includes: a plurality of sequentially connected first-type multi-layer convolutional modules, each of which is used to extract features from the forgery probability of each video frame, or to extract features from the first-layer convolutional features extracted from the connected previous first-type multi-layer convolutional module, to obtain first-layer convolutional features; at least one convolutional module, each of which is used to perform a one-dimensional convolution operation on the first-layer convolutional features extracted from the connected first-type multi-layer convolutional modules, to obtain convolutional features; and at least one second-type multi-layer convolutional module, each of which is used to extract features from the first-layer convolutional features extracted from the connected first-type multi-layer convolutional modules. The system performs feature extraction, or extracts features from upsampled features from connected upsampling modules, to obtain second multi-layer convolutional extracted features; at least one feature merger is used to perform addition operations on the second multi-layer convolutional extracted features from connected second-type multi-layer convolutional modules and the convolutional features from connected convolutional modules to obtain merged features; at least one upsampling module is used to upsample the merged features from connected feature mergers to obtain upsampled features; wherein, the plurality of target extracted features include the second multi-layer convolutional extracted features output by the second multi-layer convolutional module connected to the first multi-layer convolutional module and the upsampled features output by each of the upsampling modules.

[0018] In one possible implementation, the plurality of sequentially connected first-type multi-layer convolutional modules include a first multi-layer convolutional module, a second multi-layer convolutional module, a third multi-layer convolutional module, and a fourth multi-layer convolutional module; the at least one convolutional module includes a first convolutional module, a second convolutional module, and a third convolutional module; the at least one second-type multi-layer convolutional module includes a fifth multi-layer convolutional module, a sixth multi-layer convolutional module, and a seventh multi-layer convolutional module; the at least one feature merger includes a first feature merger, a second feature merger, and a third feature merger; and the at least one upsampling module includes a first upsampling module, a second upsampling module, and a third upsampling module. A convolutional module is used to extract features from the forgery probability of each video frame to obtain a first extracted feature; the first convolutional module is used to perform a one-dimensional convolution operation on the first extracted feature to obtain a first convolutional feature; a second multi-layer convolutional module is used to extract features from the first extracted feature to obtain a second extracted feature; the second convolutional module is used to perform a one-dimensional convolution operation on the second extracted feature to obtain a second convolutional feature; a third multi-layer convolutional module is used to extract features from the second extracted feature to obtain a third extracted feature; the third convolutional module is used to perform a one-dimensional convolution operation on the third extracted feature to obtain a third convolutional feature. The fourth multi-layer convolutional module is used to extract features from the third extracted features to obtain a fourth extracted feature; the fifth multi-layer convolutional module is used to extract features from the fourth extracted features to obtain a fifth extracted feature; the first feature merger is used to perform an addition operation on the fifth extracted feature and the third convolutional feature to obtain a first merged feature; the first upsampling module is used to upsample the first merged feature to obtain a first upsampled feature; the sixth multi-layer convolutional module is used to extract features from the first upsampled feature to obtain a sixth extracted feature; the second feature merger is used to combine the sixth extracted feature and the second convolutional feature... The first feature extraction module performs an addition operation on the second merged feature to obtain a second merged feature; the second upsampling module upsamples the second merged feature to obtain a second upsampled feature; the seventh multi-layer convolution module extracts features from the second upsampled feature to obtain a seventh extracted feature; the third feature merger performs an addition operation on the seventh extracted feature and the first convolutional feature to obtain a third merged feature; the third upsampling module upsamples the third merged feature to obtain a third upsampled feature; wherein, the plurality of target extracted features include the fifth extracted feature, the first upsampled feature, the second upsampled feature, and the third upsampled feature.

[0019] In one possible implementation, the combination module includes: an eighth multi-layer convolutional module for extracting features based on the fused features to obtain an eighth extracted feature; a fourth convolutional module for performing a one-dimensional convolution operation on the eighth extracted feature to obtain the time information of each forged segment in the video to be detected, wherein the parameters of the fourth convolutional module are set for the time information output scenario; and a fifth convolutional module for performing a one-dimensional convolution operation on the eighth extracted feature to obtain the forgery probability of each forged segment in the video to be detected, wherein the parameters of the fifth convolutional module are set for the forgery probability output scenario.

[0020] In one possible implementation, the device further includes a training module for: acquiring a training sample set, the training sample set including multiple samples, each sample including a sample video and a sample video tag corresponding to the sample video, the sample video tag being used to indicate the authenticity of the corresponding sample video; training an initial model using the training sample set until the loss function value of the model meets a preset condition to obtain a time-localization model, the loss function used for training including a one-dimensional distance intersection-union function.

[0021] In one possible implementation, the apparatus further includes a processing module for: if it is determined that the time information of multiple forged segments overlaps in time, then deleting and / or merging the time information of the multiple forged segments based on the forgery probability of each forged segment.

[0022] In one possible implementation, the device further includes a selection module for: selecting, from all forged segments, a forged segment with a forged probability greater than a preset threshold as the target result for the video to be detected.

[0023] According to another aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.

[0024] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method.

[0025] According to another aspect of this disclosure, a computer program product is provided, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0026] This disclosure obtains the detection results for each video frame in the video by detecting the authenticity of the video to be detected. The detection results represent the forgery probability of each video frame. The forgery probability of each video frame is input into the time localization model to calculate the time information of each forged segment in the video to be detected and the forgery probability of each forged segment. Without the user having to manually set a sliding window to divide the video to be detected into multiple segments, the location of the forged segment and the forgery probability in the video to be detected can be accurately obtained.

[0027] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0028] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0029] Figure 1 A schematic diagram of the BA-TFD model structure in the relevant technology is shown.

[0030] Figure 2 A flowchart illustrating the video time positioning method provided in an embodiment of this disclosure is shown.

[0031] Figure 3a This diagram illustrates the detection results of each video frame in the video to be detected provided in an embodiment of this disclosure.

[0032] Figure 3b This diagram illustrates a video authenticity detection based on a forgery detection model provided in an embodiment of this disclosure.

[0033] Figure 4 A schematic diagram of the time positioning model provided in an embodiment of this disclosure is shown.

[0034] Figure 5a A schematic diagram of a one-dimensional convolution module provided in an embodiment of this disclosure is shown.

[0035] Figure 5b A schematic diagram of a one-dimensional residual module provided in an embodiment of this disclosure is shown.

[0036] Figure 6 A block diagram of a video timing device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0037] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0038] As used herein, the terms “comprising,” “including,” “having,” or variations thereof are open-ended and include one or more of the stated features, integrals, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integrals, elements, steps, components, functions, or groups thereof.

[0039] When an element is referred to as “connected,” “coupled,” “responding,” or a variation thereof relative to another element, it may be directly connected, coupled, or responding to another element, or there may be an intermediate element present.

[0040] Although the terms first, second, third, etc., may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another. Therefore, without departing from the teachings of the inventive concept, a first element / operation in some embodiments may be referred to as a second element / operation in other embodiments.

[0041] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0042] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0043] To facilitate understanding of the technical solutions provided by the embodiments of this disclosure by those skilled in the art, the technical environment for implementing the technical solutions will be described below.

[0044] With the rapid development of artificial intelligence technologies, represented by deep learning, deepfake face technology based on models such as generative adversarial networks, variational autoencoders, and diffusion models has become increasingly sophisticated. This technology has been widely applied in scenarios such as video face-swapping, voice spoofing, and virtual digital humans. However, at the same time, the risk of its malicious misuse is gradually emerging, such as its use in creating fake videos, political manipulation, and defamation, posing a serious threat to social security.

[0045] With the development of deepfake generation technology, the work of detecting deepfake faces has also gradually emerged. The detection targets have evolved from the poor-quality videos generated in the early days of deepfakes to the forged videos that are now difficult to distinguish with the naked eye. The quality of the forged video frame images is not only getting higher and higher, but the consistency between the original and the forged images is also getting better and better.

[0046] However, current mainstream detection methods cannot accurately pinpoint the time segment in which the forgery occurred, limiting their practicality in scenarios such as source tracing and accountability. With the continuous advancement of research into deepfake face detection technology, many methods have achieved good results in classifying video authenticity. However, existing methods generally implicitly assume that the entire forged video is modified, meaning that any segment of the video taken is a forgery. This assumption holds true on datasets such as FF++ and Celeb-DF(v2), therefore researchers have long focused more on the model's ability to classify deepfake face videos from real videos. But now, forgers can carefully replace parts of the video, altering local content while retaining most of the original content to increase credibility and reduce the probability of detection. For example, in a science video, if a scientist says "vaccines are safe," a forger can simply replace "safe" with "dangerous," thus subverting the entire message conveyed by the video.

[0047] The task of spoofing time localization is to locate the start and end times of spoofed segments in videos that may contain localized spoofed content. Compared to deepfake face detection, this task is significantly more challenging. Current research on spoofing time localization is relatively limited; most studies still use a sliding window approach to segment the video into multiple segments and then apply detection methods to each segment to determine the start and end times. However, this approach has two drawbacks: first, it involves a large computational load, as the same video frames may need to be calculated multiple times using different windows, which is clearly unnecessary additional computation; second, it is highly sensitive to the choice of window size, which increases the number of hyperparameters that users need to configure.

[0048] LAV-DF also proposes a Boundary-Aware Temporal Forgery Detection (BA-TFD) model, which is a 3D convolutional neural network (CNN)-based model for tampering with time. Its model structure is as follows: Figure 1As shown, the corresponding method involves extracting multi-frame image features using a 3D CNN network as a video encoder, and processing the Mel-spectrum input using a 2D CNN network. Then, a contrastive loss is used to treat audio-video feature pairs from real videos as positive sample pairs, minimizing the feature distance between them; audio-video feature pairs from videos containing fake segments are treated as negative sample pairs, ensuring that the distance between them is greater than a preset value. Based on this, the model gains the ability to detect the authenticity of audio and video. Furthermore, this method also uses frame classification loss, directly comparing the encoder's output with the labels to help the encoder acquire more accurate features. The boundary matching loss proposed in this study is key to achieving fake time localization. The model predicts a boundary map, i.e., the confidence level of all possible segments in the video with different durations and start times, indicating whether they are fake segments.

[0049] Existing methods for temporal localization of deepfake faces perform reasonably well on simple deepfake videos, but their boundary matching loss method performs poorly on high-quality deepfake videos with shorter segments. Furthermore, current boundary loss-based methods lack universality in detecting fake segments of varying lengths, making them difficult to handle in practice with segments of significantly different lengths. Additionally, most existing methods divide the video into multiple overlapping segments according to a fixed window size, then classify these segments separately using existing deepfake detection methods, requiring manual design of parameters such as window size, resulting in low accuracy.

[0050] To address the aforementioned technical issues, this disclosure provides a video time localization method that can accurately detect the time information and forgery probability of forged segments in deepfake videos. It has the ability to detect forged segments of different lengths and does not require the user to manually set a sliding window to divide the video to be detected into multiple segments.

[0051] Now combined Figure 2 Figure 5 illustrates the video timing method provided in this embodiment of the present disclosure. Figure 2 As shown, this method may include the following steps S101 and S102.

[0052] Step S101: Detect the authenticity of the video to be detected and obtain the detection results for each video frame in the video to be detected.

[0053] The detection result represents the forgery probability of a video frame. This method, based on the forgery probability of video frames, can complete the forgery time localization task, which refers to locating the start and end times of forged segments in a video where forgery is possible. This method requires preprocessing the original video to be detected. Preprocessing can involve detecting the authenticity of the video to save the forgery probability of each video frame. The class of the video frame can be determined by the magnitude of the forgery probability; the class can be real or forged. Figure 3a In the algorithm, if the probability of a video frame being forged is 0.12, the video frame is determined to be real; if the probability of a video frame being forged is 0.99, the video frame is determined to be forged.

[0054] Step S101 may include the process of authenticity detection for the video to be detected: extracting each target frame pair and the first audio feature for each target frame pair from the video to be detected. Each target frame pair includes two adjacent video frames and both adjacent video frames include the same target object. A video frame is the smallest time unit that constitutes a video. In other words, a video frame is a single static image. The first audio feature includes Mel-spectrum information. Each target frame pair and the first audio feature for that target frame pair are input into a forgery detection model (hereinafter referred to as the detection model) for calculation to obtain a score for each target frame pair. The score represents the forgery probability of the target frame pair. Based on the score of each target frame pair, the detection result for each video frame in the video to be detected is determined. For example, the forgery probability of the target frame pair can be directly used as the forgery probability of each of the two video frames in that target frame pair.

[0055] The process of detecting the authenticity of a video can also include data preprocessing and data augmentation. Taking a video that is a potential deepfake face video as an example, data preprocessing can include: first, extracting the keyframes of the video into multiple original images according to the frame rate (Frames Per Second, FPS); since different videos have different perspectives and the proportion of the face region in the frame, in order to unify the input, this method uses dlib, a machine learning algorithm library widely used in academia and industry, as a face extractor to extract the face region from each original image and save it as the corresponding face image; then, each pair of adjacent face images is taken as a target frame pair, and the same data augmentation is performed on the two face images in this target frame pair. Data augmentation may include the following steps: resizing, such as resizing a face image to 256*256 pixels; center cropping, such as cropping a central region of 224*224 pixels; horizontal flipping, which can be done randomly, such as flipping the cropped image with a preset probability of 0.5; rotation, which can be done randomly, such as rotating the horizontally flipped image with a preset probability of 0.5, with the rotation angle being ±30°; random brightness and contrast adjustment, such as adjusting the rotated image with a preset probability of 0.5, with the adjustment amount being ±0.2; and color jitter adjustment, which can be done randomly, such as adjusting the color jitter of the randomly brightened and contrasted image with a preset probability of 0.5. The image undergoes color dithering adjustment, with saturation and hue adjustments ranging from ±0.2. Gaussian blur adjustment can be performed randomly, for example, with a preset probability of 0.3 on the color-dithered image, and the blur radius can be set between 3 and 7. Gaussian noise adjustment can also be performed randomly, for example, with a preset probability of 0.3 on the Gaussian-blurred image, and the variance can be set between 10 and 50. Standardization is then performed, specifically standardizing the Gaussian-noise-adjusted image to RGB values. Finally, these RGB values ​​are converted into tensors for processing by the neural network framework, thus serving as image features input to the detection model. The order of horizontal flipping, rotation, random brightness and contrast adjustment, color dithering adjustment, Gaussian blur adjustment, and Gaussian noise adjustment in data augmentation can be adjusted according to actual needs, and this disclosure does not impose specific limitations on this.

[0056] In the process of detecting the authenticity of a video, to extract the audio features corresponding to the target frame pair, the audio track in the video is first extracted and saved, and then its Mel Spectrum feature is extracted. Mel Spectrum is a commonly used feature method in speech signal processing, which transforms the frequency of the speech signal to a nonlinear frequency scale that is closer to the human ear's auditory perception. Because the human ear is more sensitive to low frequencies and relatively less sensitive to high frequencies, the Mel frequency scale uses a nonlinear transformation to suppress the output of high-frequency signals.

[0057] For example, suppose the audio signal extracted from the video to be detected is a discrete signal. ,in This indicates the number of sampling points, and the sampling frequency is... (Unit: Hz). To convert the time-domain signal to the frequency domain for analysis, a Short-Time Fourier Transform (STFT) is required to segment the audio signal. The length of each frame is expressed as... The window movement step size is Each frame of audio signal refers to a sequence of consecutive audio signals. According to a fixed length The segmented short-segment signal is obtained. The audio signal extraction for each frame can be represented by the following equation 1:

[0058] Formula 1

[0059] In Equation 1, It is the frame index. Indicates the first The audio signal of the frame, It is a length of The window function (such as the Hamming window) is used, and the explanations of the other parameters are given above, so they will not be repeated here. Then, for each frame... The Discrete Fourier Transform (DFT) is calculated, and the calculation process can be expressed as Equation 2 below:

[0060] Formula 2

[0061] In Equation 2, Indicates the first Frame in frequency index The spectrum on This indicates the length of the frame (also the number of points calculated by the DFT). The value represents the imaginary unit; the meanings of the other parameters are explained above and will not be repeated here. The power spectrum is then calculated based on the DFT results. The process of calculating the power spectrum can be expressed as Equation 3:

[0062] Formula 3

[0063] In Equation 3, Indicates the first Frame in frequency index The power spectrum is shown above; the explanations of the other parameters are provided above and will not be repeated here. Thus, based on... A typical signal frequency spectrum can be obtained, and then this frequency spectrum can be mapped to a Mel frequency scale. Specifically, the Mel filter bank can be applied to this frequency spectrum according to the following equation 4:

[0064] Formula 4

[0065] In Equation 4, The frequency is linear. This process converts the signal from a linear scale to a Mel frequency scale, and its main function is to suppress the high-frequency response. Next, a logarithmic transformation is performed on the energy output of the Mel filter, which can be expressed as Equation 5:

[0066] Formula 5

[0067] In Equation 5, It is the first Frame in Energy on a Mel filter It is a very small positive number, used to avoid overflow problems caused by the logarithm tending towards negative infinity. Finally, the Mel spectrum is normalized and used as the first audio feature, which includes Mel spectrum information. The normalization process can be expressed as Equation 6:

[0068] Formula 6

[0069] In Equation 6, This represents the normalized Mel spectrum, which is the first audio feature. The unnormalized Mel spectrum represents the result of performing a logarithmic transformation on the energy output of the Mel filter. express The Euclidean norm, This indicates the number of Mel filters.

[0070] The detection model predicts a score for each target frame pair based on the input target frame pair and a first audio feature specific to that target frame pair. The score represents the probability that the target frame pair is a forgery. For example, the closer the score of a target frame pair is to 1, the higher the probability that the target frame pair is a forgery. Figure 3bAs shown, the detection model may include a video module, an audio-video module, and a linear combiner. During the testing phase, the detection model predicts the score of the target frame pair using the video module, audio-video module, and linear combiner. The video module can be used to determine a first classification result based on the target frame pair, which represents the forgery probability of the target frame pair. The audio-video module can be used to determine a second classification result based on the target frame pair and a first audio feature for that target frame pair, which represents the forgery probability of the target frame pair. The linear combiner (LC) can be used to determine the score of the target frame pair based on a preset first weight matrix associated with the video module, the first classification result, a preset second weight matrix associated with the audio-video module, and the second classification result. For example, the linear combiner can calculate the score of the target frame pair using the following equation 7:

[0071] Formula 7

[0072] In Equation 7, This represents the score of the target frame pair. This represents the second weight matrix related to the audio and video modules. This indicates the second classification result output by the audio / video module. This represents the first weight matrix related to the video module. This represents the first classification result output by the video module. The specific first and second weight matrices can be obtained by training the detection model.

[0073] The video module is responsible for detecting intra-frame and inter-frame inconsistencies between two video frames of a target frame pair. For example... Figure 3b As shown, the video module may include a first image encoder, a first feature comparator, and a first classifier. The first image encoder can be used to extract features from two adjacent video frames of the target frame pair to obtain first extracted features. The first feature comparator can be used to compare features based on the first extracted features of the two adjacent video frames of the target frame pair to obtain first difference features. For example, the first difference features can be obtained by subtracting the first extracted features of these two video frames. The first difference features can represent the difference information between the two adjacent video frames of the target frame pair in the image feature space. The first classifier can be used to determine a first classification result based on the first difference features.

[0074] The audio / video module is responsible for detecting cross-modal inconsistencies between two video frames of a target frame pair and between the audio segments of those two video frames. For example... Figure 3bAs shown, the audio / video module may include a second image encoder, a second feature comparator, an audio encoder, a cross-attention layer, and a second classifier. The second image encoder can be used to extract features from two adjacent video frames of the target frame pair, respectively, to obtain second extracted features. The second feature comparator can be used to compare features based on the second extracted features of two adjacent video frames of the target frame pair, to obtain second difference features. For example, the second difference features can be obtained by subtracting the second extracted features of these two video frames. The second difference features represent the difference information between two adjacent video frames of the target frame pair in the image feature space. The audio encoder can be used to extract features from the first audio features, to obtain second audio features. The cross-attention layer can be used to fuse the second difference features and the second audio features, to obtain fused features. The second classifier can be used to determine a second classification result based on the fused features.

[0075] The detection model may also include a target teacher model. The target teacher model is responsible for correcting video-level labels to correct frame-level labels and assisting in the training of the first classifier in the video module. The training process of the detection model may include: acquiring a first training sample set, which includes multiple first samples. Each first sample includes a first sample frame pair, sample audio features for that first sample frame pair, and a first video label for that first sample frame pair. The first video label is the label of the video containing the first sample frame pair, and it is used to indicate the authenticity of the corresponding video. The first video label is either real or fake (or false). The first samples in the first training sample set come from real and fake videos. When constructing each first sample, the data preprocessing and data augmentation described above can also be used to process the video containing the first sample frame pair. Each first sample frame pair includes... The first sample frame includes two adjacent video frames, both of which contain the same sample object. The sample object can be flexibly selected according to actual needs. The audio features of the sample include the Mel spectrum information between the two video frames of the first sample frame pair. The initial detection model is trained using the first training sample set to obtain the trained detection model. Specifically, in this detection method, the classification results of the video module and the audio-visual module are weighted and averaged by a linear combiner with learnable parameters and compared with the first video label. The detection model is trained using the gradient backpropagation method. The parameters of the first classifier of the video module can be adjusted by correcting the frame label obtained by the target teacher model.

[0076] The target teacher model can be used to obtain frame labels based on the first video label and the first sample frame pair, and to adjust the parameters of the first classifier in the video module of the detection model based on the frame labels. For example... Figure 3bAs shown, the target teacher model may include a third image encoder, a third feature comparator, a third classifier, and a corrector. The third image encoder can be used to extract features from two adjacent video frames of the first sample frame pair, obtaining third extracted features. The third feature comparator can be used to compare features based on the third extracted features of two adjacent video frames of the first sample frame pair, obtaining third difference features. For example, the third difference features can be obtained by subtracting the third extracted features of these two video frames. The third difference features represent the difference information between two adjacent video frames of the first sample frame pair in the image feature space. The third classifier can be used to determine the third classification result based on the third difference features. The third classification result represents the forgery probability of the first sample frame pair. The corrector can be used to determine the frame label for the first sample frame pair based on the third classification result, the first video label, and the first sample frame pair. The frame label represents the authenticity of the video containing the corresponding first sample frame, which may differ from the first video label of the first sample frame pair. This detection method features a label correction mechanism that provides frame-level realism labels through a target teacher model. This helps the detection model train efficiently even when only video-level realism labels are available and only local segments of the video are fake. This allows the model to focus more effectively on fake features rather than the features of the dataset itself, thus avoiding overfitting to the dataset.

[0077] Before training the video module, audio-video module, and linear combiner in the detection model, a target teacher model is first trained. The training process of the teacher model may include: acquiring a second training sample set, which includes multiple second samples. Each second sample includes a pair of second sample frames and a second video label for that pair. The second video label is the label of the video containing the second sample frame pair, indicating the authenticity of the corresponding video and that the authenticity of the second video label indicates forgery. In other words, all second samples in the second training sample set come from forged videos. Each pair of second sample frames includes two adjacent video frames, and these two adjacent video frames include the same sample object. The sample object can be flexibly selected according to actual needs. The initial teacher model is then trained using the second training sample set to obtain the target teacher model. Specifically, the second training sample set can be the FF++ dataset from related technologies. The FF++ dataset is chosen as the training dataset for the teacher model for two reasons: first, all video frames in the deepfake videos in the FF++ dataset are forged frames, eliminating the need to consider label inconsistency issues; second, the FF++ dataset is a unimodal dataset and does not have audio forgery issues. For example, the initial teacher model can be a video module including a backbone network and a classification network. The backbone network can be, but is not limited to, a ResNet-18 model, and the classification network can be, but is not limited to, a convolutional neural network. The video module including the backbone network and the classification network is trained directly on the FF++ dataset, and the trained model parameters are saved and used as the parameters of the target teacher model. When training the video module, audio-video module, and linear combiner in the detection model, the parameters of the target teacher model are frozen, specifically the parameters of the third image encoder and the third classifier.

[0078] During the training of the detection model, the first sample frame pair includes and , Indicates the first The image of the frame, Indicates the first The input image of the frame. First, feature extraction is performed by three image encoders (i.e., the first image encoder, the second image encoder, and the third image encoder) to obtain the corresponding extracted features. Then, feature comparison is performed by the feature comparators corresponding to these three image encoders (i.e., the first feature comparator, the second feature comparator, and the third feature comparator) to obtain the image features extracted from the video module, the audio-visual module, and the target teacher model. These image features can be expressed as Equation 8:

[0079] Formula 8

[0080] In Equation 8, This represents the image features extracted by the target teacher model (i.e., the third difference feature output by the third feature comparator). This represents the third image encoder in the target teacher model. Indicates the third image encoder from The third extracted feature is obtained from it. Indicates the third image encoder from The third extracted feature is obtained from it.

[0081] In Equation 8, This represents the image features extracted from the video module (i.e., the features output by the first feature comparator). This indicates the first image encoder in the video module. Indicates the first image encoder from The features extracted from them Indicates the first image encoder from The features extracted are used by the first image encoder to extract features from the input video frames during both the testing and training processes. The first feature comparator is used to compare the features of the two video frames output by the first image encoder during both the testing and training phases to obtain the corresponding difference features.

[0082] In Equation 8, This represents the image features extracted by the audio / video module (i.e., the features output by the second feature comparator). This refers to the second image encoder in the audio / video module. Indicates the second image encoder from The features extracted from them Indicates the image encoder from The features extracted are used by the second image encoder to extract features from the input video frames during both the testing and training phases. The second feature comparator is used to compare the features of the two video frames output by the second image encoder during both the testing and training phases to obtain the corresponding difference features.

[0083] , , These three features represent the differences between consecutive frames in the image feature space within the first sample frame pair, including both intra-frame and inter-frame inconsistencies. The first, second, and third image encoders can all use the ResNet-18 model as their backbone network. and Participate in gradient update, The parameters are frozen.

[0084] During the training process of the detection model, the first sample frame pair and The corresponding sample audio features (denoted as) The audio encoder (denoted as) of the audio / video module Audio features are obtained by feature extraction. .

[0085] After extracting image and audio features respectively, the corrected frame labels are calculated using the target teacher model. The calculation process can be expressed as Equation 9:

[0086] Formula 9

[0087] In Equation 9, This represents the frame label output after correction by the target teacher model. This represents the function that takes the minimum value. This represents the third classifier in the target teacher model (the parameters of this classifier are also frozen and do not participate in gradient updates). Indicates the third difference characteristic. This represents the first video label, where 0 represents a real label and 1 represents a fake label. If the first video label is real, then all video frames in the video must be real, and the frame-level labels output by the target teacher model using the minimum value strategy will also be real. If the first video label is fake, then the video frames in the video may be real or fake. In this case, the frame-level labels output by the target teacher model using the minimum value strategy are the results of the third classification model, which can solve the problem of inconsistency between video-level labels and frame-level labels caused by the fakeness of local segments.

[0088] Image features extracted by the audio and video module and audio features After feature fusion through the multi-head cross-attention layer, the fused features are obtained. This fusion process can be expressed as Equation 10:

[0089] Formula 10

[0090] In Equation 10, This represents the fused features output by the cross-attention layer. This represents the calculation of the multi-head cross-attention layer. This represents the feature output by the second feature comparator. This represents the audio features extracted by the audio encoder from the sample audio features.

[0091] The forgery probability of the first sample frame pair output by the audio / video module is obtained through the second classifier of the audio / video module. The forgery probability of the first sample frame pair output by the video module is obtained through the first classifier of the video module. Finally, the score of the first sample frame pair is calculated through a linear combiner, which can be expressed as Equation 11 below:

[0092] Formula 11

[0093] In Equation 11, This represents the score of the first sample frame pair output by the linear combiner. This represents the weight matrix related to the audio and video modules. This represents the forgery probability of the first sample frame pair output by the second classifier in the audio / video module. This represents the weight matrix related to the video module. This represents the forgery probability of the first sample frame pair output by the first classifier in the video module, where... and It is a learnable weight matrix, which can be obtained by training the detection model. and .

[0094] The loss function used in the training of the detection model can be calculated using the binary cross-entropy (BCE) method, which can be expressed as Equation 12 below:

[0095] Formula 12

[0096] In Equation 12, This represents the loss function; the meanings of the other parameters are explained above and will not be repeated here. Training of the detection model can be stopped when the loss function value is less than a preset threshold.

[0097] During the testing phase, i.e., when actually detecting the authenticity of the video to be tested, all parameters of the video module, audio / video module, and linear combiner are frozen. Multiple target frame pairs from the video to be tested are input into the trained detection model for calculation, which yields a score for each target frame pair, i.e., the forgery probability. Then, based on the scores of each target frame pair, the detection result for each video frame in the video to be tested is determined. Specifically, the forgery probability of a target frame pair can be directly used as the forgery probability of each of the two video frames within that target frame pair.

[0098] In fact, other methods in related technologies can be used to detect the authenticity of the video to be tested, as long as the forgery probability of each video frame in the video to be tested is obtained.

[0099] Step S102: Input the forgery probability of each video frame into the time localization model for calculation to obtain the time information of each forged segment of the video to be detected and the forgery probability of each forged segment.

[0100] By analyzing the timing information of the forged segment, we can determine its specific location within the video to be detected, specifically its start and end times. The probability of the forged segment being genuine indicates the likelihood of it being a forgery.

[0101] The key difference between temporal localization of deepfake videos and traditional object detection lies in the fact that traditional object detection uses two-dimensional rectangles for anchor boxes to determine the detection result, while temporal localization is performed in one-dimensional space. Therefore, the temporal localization model based on a one-dimensional region proposal network used in this method outputs one-dimensional bounding boxes, and the output classification result is only "real" or "fake." The one-dimensional bounding box corresponds to a one-dimensional segment, i.e., a time segment. Based on this, this method proposes a temporal localization method for fake segments in videos based on a temporal localization model. During the calculation process of the temporal localization model, such as... Figure 4 As shown, features of different sizes are first extracted through multiple one-dimensional convolutions, then the features are fused through upsampling and attention layers, and finally one-dimensional regression boxes and corresponding classification scores are output. The one-dimensional regression boxes are used to indicate the time information of the forged segment, and the normalized classification scores can represent the forgery probability of the forged segment. The number of one-dimensional regression boxes depends on the actual situation of the video to be detected.

[0102] The temporal localization model can include a feature extraction module, an attention layer, and a combination module. The feature extraction module can perform multiple feature extractions and upsampling based on the forgery probability of each video frame to obtain multiple target-extracted features. The forgery probability of each video frame can be a one-dimensional matrix or a matrix of different dimensions converted from the last layer features output by the previous network (such as the forgery detection model mentioned earlier). The attention layer can perform feature fusion based on multiple target-extracted features to obtain fused features. The combination module can determine the temporal information of each forged segment and the forgery probability of each forged segment based on the fused features. This method proposes an attention fusion mechanism based on multi-size features, automatically training feature fusion weights of different network layers using the attention mechanism. Compared with traditional object detection models, it achieves better fusion results for features of different sizes and has a stronger ability to capture long-distance dependencies.

[0103] The feature extraction module may include: multiple first-class multi-layer convolutional modules connected in sequence, each first-class multi-layer convolutional module being used to extract features from the forgery probability of each video frame, or to extract features from the first-class multi-layer convolutional features extracted from the connected previous first-class multi-layer convolutional module, to obtain first-class multi-layer convolutional features; at least one convolutional module, each convolutional module being used to perform one-dimensional convolution operations on the first-class multi-layer convolutional features extracted from the connected first-class multi-layer convolutional modules, to obtain convolutional features; at least one second-class multi-layer convolutional module, each second-class multi-layer convolutional module being used to extract features from the first-class multi-layer convolutional features extracted from the connected first-class multi-layer convolutional modules, or to extract features from the upsampled features extracted from the connected upsampling module, to obtain second-class multi-layer convolutional features; at least one feature merger, each feature merger being used to perform addition operations on the second-class multi-layer convolutional features extracted from the connected second-class multi-layer convolutional modules and the convolutional features from the connected convolutional modules, to obtain merged features; and at least one upsampling module, each upsampling module being used to upsample the merged features from the connected feature merger, to obtain upsampled features. Among them, the multiple target extraction features include the second multi-layer convolution extraction features output by the second multi-layer convolution module connected to the first multi-layer convolution module, and the upsampling features output by each upsampling module.

[0104] The feature extraction module may include a first multi-layer convolutional module, a second multi-layer convolutional module, a third multi-layer convolutional module, and a fourth multi-layer convolutional module, which can be specifically identified as follows: Figure 4 In the M1 to M4 feature extraction modules, at least one second-type multilayer convolutional module may include a fifth, sixth, and seventh multilayer convolutional module, which can be sequentially represented as follows: Figure 4 M5 to M7 in the feature extraction module. At least one convolutional module in the feature extraction module may include a first convolutional module, a second convolutional module, and a third convolutional module, which can specifically correspond to M5 to M7 in sequence. Figure 4 C1 to C3 in the above. At least one feature merger in the feature extraction module may include a first feature merger, a second feature merger, and a third feature merger, which can specifically correspond to the following in sequence: Figure 4 A1 to A3 in the diagram. At least one upsampling module in the feature extraction module may include a first upsampling module, a second upsampling module, and a third upsampling module, which can be specifically identified as follows: Figure 4 U1 to U3 in the series.

[0105] The first multi-layer convolutional module can be used to extract features from the forgery probability of each video frame, obtaining the first extracted feature. The first convolutional module can also perform a one-dimensional convolution operation on the first extracted feature, obtaining the first convolutional feature. The second multi-layer convolutional module can be used to extract features from the first extracted feature, obtaining the second extracted feature. The second convolutional module can also perform a one-dimensional convolution operation on the second extracted feature, obtaining the second convolutional feature. The third multi-layer convolutional module can be used to extract features from the second extracted feature, obtaining the third extracted feature. The third convolutional module can also perform a one-dimensional convolution operation on the third extracted feature, obtaining the third convolutional feature. The fourth multi-layer convolutional module can be used to extract features from the third extracted feature, obtaining the fourth extracted feature. The fifth multi-layer convolutional module can be used to extract features from the fourth extracted feature, obtaining the fifth extracted feature. The first feature merger can perform an addition operation on the fifth extracted feature and the third convolutional feature, obtaining the first merged feature. The first upsampling module can upsample the first merged feature, obtaining the first upsampled feature. The sixth multi-layer convolutional module can extract features from the first upsampled feature, obtaining the sixth extracted feature. The second feature merger can be used to add the sixth extracted feature and the second convolutional feature to obtain the second merged feature. The second upsampling module can be used to upsample the second merged feature to obtain the second upsampled feature. The seventh multi-layer convolutional module can be used to extract features from the second upsampled feature to obtain the seventh extracted feature. The third feature merger can be used to add the seventh extracted feature and the first convolutional feature to obtain the third merged feature. The third upsampling module can be used to upsample the third merged feature to obtain the third upsampled feature. The multiple target extracted features (i.e., the extracted features from the input attention layer) include the fifth extracted feature, the first upsampled feature, the second upsampled feature, and the third upsampled feature.

[0106] The combined module may include an eighth multi-layer convolutional module, a fourth convolutional module, and a fifth convolutional module, which can be sequentially represented as follows: Figure 4 The M8, C4, and C5 modules are used for feature extraction based on the fused features. The eighth multi-layer convolutional module can be used to extract features based on the fused features, resulting in the eighth extracted features. The fourth convolutional module can be used to perform one-dimensional convolution operations on the eighth extracted features to obtain the temporal information of each forged segment in the video to be detected. The parameters of the fourth convolutional module are set for the temporal information output scenario. The fifth convolutional module can be used to perform one-dimensional convolution operations on the eighth extracted features to obtain the forgery probability of each forged segment in the video to be detected. The parameters of the fifth convolutional module are set for the forgery probability output scenario.

[0107] The M1 to M8 multi-layer convolutional modules in the temporal localization model can include one-dimensional convolutional modules (Conv1dBlock) and one-dimensional residual modules (ResBlock1d). For example, as... Figure 5aAs shown, a one-dimensional convolutional module may include a one-dimensional convolutional layer (Conv1d), a one-dimensional normalization layer (BatchNorm1d), and a corrected linear / activation layer (ReLU); as Figure 5b As shown, the one-dimensional residual module may include a one-dimensional convolutional layer (Conv1d), a one-dimensional normalization layer (BatchNorm1d), a one-dimensional convolutional layer (Conv1d), a one-dimensional normalization layer (BatchNorm1d), and an adder arranged sequentially.

[0108] This method may further include: acquiring a training sample set, which includes multiple samples, each sample including a sample video and a corresponding sample video label. The sample video label is used to indicate the authenticity of the corresponding sample video, and the sample video label is real or fake. In other words, a sample can be a segment labeled as real or fake (with an anchor box), and a sample can be a positive sample or a negative sample. A positive sample is fake / abnormal, and a negative sample is normal. An initial model is trained using the training sample set until the model's loss function value meets a preset condition, resulting in a time-localization model. The loss function used for training includes the one-dimensional distance intersection over union (D-DIoU) function. Specifically, during the training and validation of the model, segments with pre-set anchor boxes are first input into the model to extract features and provide classification scores and regression boxes. The normalized classification score can be used as the classification probability, and the regression box can be used to indicate the center position and width.

[0109] The 1D-DIoU function combines the design principles of both Focal Loss and DIoU Loss. Focal Loss is a loss function that applies different weights to positive and negative samples, often used in object detection models based on region proposal networks. By adjusting the weights, the weight of positive samples (i.e., outlier fragments) can be increased, making the model focus more on classifying positive samples during training. DIoU Loss, on the other hand, considers the distance between the center of the predicted bounding box and the ground truth center in addition to the normal regression loss. In the one-dimensional case, it is equivalent to adding a second-order regression loss on top of the ordinary regression loss.

[0110] In this method, the loss function used to train the time-localization model can be expressed as Equation 13:

[0111] total_loss = *cls_loss + *reg_loss Formula 13

[0112] In Equation 13, total_loss represents the value of the loss function. This represents the first weight coefficient corresponding to the classification prediction, and cls_loss represents the classification prediction loss. represents the second weight coefficient corresponding to the regression prediction, reg_loss represents the regression prediction loss, and cls_loss and reg_loss are both calculated through the time positioning model.

[0113] The classification prediction loss cls_loss can be expressed as the following equation 14:

[0114] cls_loss = ∑(pos_loss+neg_loss) / (batch_size + Formula 14

[0115] In Equation 14, pos_loss represents the positive sample loss; pos_loss = per_sample_loss1 * pos_mask, where per_sample_loss1 represents the loss for each positive sample, and pos_mask represents the positive sample mask; neg_loss represents the negative sample loss; neg_loss = per_sample_loss2 * neg_mask, where per_sample_loss2 represents the loss for each negative sample, and neg_mask represents the negative sample mask; batch_size represents the batch size. This represents a very small value to avoid division by zero errors. Equation 14 is used to calculate the classification prediction loss value for a batch of samples, and Equation 15 is used similarly to calculate the regression prediction loss value for a batch of samples.

[0116] The regression prediction loss reg_loss can be expressed as Equation 15:

[0117] reg_loss=∑(diou_loss*pos_mask) / (∑pos_mask + Formula 15

[0118] In Equation 15, diou_loss represents the core loss value, used to quantify the regression bias between regression prediction and classification prediction. The other parameters are described above and will not be repeated here. diou_loss = 1 – diou, where diou represents the normalized distance intersection-union ratio. diou can be expressed as Equation 16:

[0119] diou=iou–center_distance 2 / (enclosing_length 2 + Formula 16

[0120] In Equation 16, IOU represents the Intersection over Union (IoU), which is calculated by dividing the intersection of the output bounding box (or result fragment) and the true value by the union of the two. It is a commonly used evaluation metric. IoU ranges from 0 to 1; a larger IoU indicates more accurate prediction. IoU = inter_length / (union + ... ), inter_length represents the length of the intersection, union represents the length of the union, enclosing_length represents the length of the minimum enclosing interval, center_distance represents the center distance (specifically, the distance between the center position of the regression prediction box and the actual fake fragment), and center_distance can be expressed as the following formula 17:

[0121] center_distance=|reg_preds.center–reg_targets.center| Formula 17

[0122] In Equation 17, reg_preds.center represents the center position of the regression prediction box, and reg_targets.center represents the center position of the actual fake fragment.

[0123] For example, during the training of the temporal localization model, the loss function value can be obtained according to the calculation process shown in Table 1:

[0124] Table 1. Calculation process of 1D-DIoU loss function

[0125] In machine learning model training, data sampling refers to the process of selecting a subset of data from the original training sample set according to specific rules as training or validation data. Its core purpose is to optimize data distribution and improve model efficiency and generalization ability. In the data sampling process of this method, a one-dimensional adaptive training sample selection (1D-ATSS) strategy is adopted. Its core idea is to use a dynamic IoU threshold to ensure that each positive sample has a corresponding prediction segment for training. This helps the temporal localization model obtain enough positive samples in the early stage, avoiding the filtering out of all prediction segments due to not reaching the rigid IoU threshold, and balancing the ratio of positive to negative samples.

[0126] This method trains a temporal localization model using 1D-ATSS, enabling automatic selection of positive and negative samples based on the statistical characteristics of the target. Specifically, for each target segment (the bounding box corresponding to the real fake segment), k anchor boxes closest to the center are selected as candidate positive samples; the IoU values ​​between these candidate anchor boxes and the target box are calculated, thereby calculating the mean of these IoU values. and standard deviation And set the IoU threshold to t = + Finally, anchor boxes with an IoU value ≥ t and whose centers are within the target bounding box are selected as positive samples. Thus, training the temporal localization model with 1D-ATSS enables efficient and robust sample selection with almost no additional hyperparameters.

[0127] For example, during the training of the time localization model, the 1D-ATSS strategy shown in Table 2 can be used for training:

[0128] Table 2 1D-ATSS Strategy

[0129] This method may further include: if it is determined that the time information of multiple forged segments overlaps, then the time information of the multiple forged segments is reduced and / or merged based on the forgery probability of each forged segment. For example, if the time information of forged segment 1 includes a start time t1 and an end time t2, and the time information of forged segment 2 includes a start time t2 and an end time t3, where t1 is earlier than t2 and t2 is earlier than t3, then it is determined that the time information of forged segment 1 and forged segment 2 overlaps. Therefore, based on the forgery probability y1 of forged segment 1 and the forgery probability y2 of forged segment 2, where both y1 and y2 are greater than a preset forgery threshold y... th Then the time information of forged fragment 1 and forged fragment 2 is merged, where y th Set appropriate values ​​according to the actual situation. For example, if the time information of forged segment 3 includes a start time t1 and an end time t3, and the time information of forged segment 4 includes a start time t2 and an end time t4, where t1 is earlier than t2, t2 is earlier than t3, and t3 is earlier than t4, then it is determined that the time information of forged segment 3 and forged segment 4 overlaps. If the forgery probability y3 of forged segment 3 is greater than the forgery probability y4 of forged segment 4, and y3 is greater than y4... th And y4 is less than y th Then, the time information from t2 to t3 in forged segment 3 will be deleted. The process of deleting and merging the time information of multiple forged segments is the same as the example above, and will not be repeated here.

[0130] This method may further include: selecting, from all forged segments, forged segments with a forged probability greater than a preset threshold as the target result for the video to be detected. The preset threshold is selected as an appropriate value based on the actual situation to obtain the target result that best matches the actual situation, thereby identifying the forged segments in the time frame to be detected.

[0131] This method offers numerous advantages over existing approaches. It utilizes a temporal localization model to output the forgery probability and temporal information (start and end times) of forged segments. Furthermore, it adapts to varying forged segment lengths, effectively detecting even very short forged segments. Moreover, it eliminates the need for users to manually set sliding windows to segment the video, and its localization performance significantly surpasses existing methods. Compared to previous forgery time localization methods employing sliding window classification, this method offers a more unified approach. It eliminates the need for prior statistical analysis of the dataset to determine the sliding window size and post-processing of the classification sequences generated by the sliding window. Instead, the model directly outputs the temporal information and forgery probability of the forged segments, making it more efficient and flexible.

[0132] This disclosure also provides a video time positioning device, the device comprising: a detection module, configured to detect the authenticity of a video to be detected and obtain detection results for each video frame in the video to be detected, the detection results representing the forgery probability of the video frame; and a calculation module, configured to input the forgery probability of each video frame into a time positioning model for calculation, to obtain time information of each forged segment of the video to be detected and the forgery probability of each forged segment.

[0133] In one possible implementation, detecting the authenticity of the video to be detected and obtaining detection results for each video frame in the video to be detected includes: extracting each target frame pair and a first audio feature for each target frame pair from the video to be detected, wherein each target frame pair includes two adjacent video frames, and the first audio feature includes Mel-spectrum information; inputting each target frame pair and the first audio feature for each target frame pair into a forgery detection model for calculation to obtain a score for each target frame pair, wherein the score represents the forgery probability of the target frame pair; and determining the detection result for each video frame in the video to be detected based on the score of each target frame pair.

[0134] In one possible implementation, the time localization model includes: a feature extraction module, used to perform multiple feature extractions and multiple upsamplings based on the forgery probability of each video frame to obtain multiple target extracted features; an attention layer, used to perform feature fusion based on the multiple target extracted features to obtain fused features; and a combination module, used to determine the time information of each forged segment and the forgery probability of each forged segment based on the fused features.

[0135] In one possible implementation, the feature extraction module includes: a plurality of sequentially connected first-type multi-layer convolutional modules, each of which is used to extract features from the forgery probability of each video frame, or to extract features from the first-layer convolutional features extracted from the connected previous first-type multi-layer convolutional module, to obtain first-layer convolutional features; at least one convolutional module, each of which is used to perform a one-dimensional convolution operation on the first-layer convolutional features extracted from the connected first-type multi-layer convolutional modules, to obtain convolutional features; and at least one second-type multi-layer convolutional module, each of which is used to extract features from the first-layer convolutional features extracted from the connected first-type multi-layer convolutional modules. The system performs feature extraction, or extracts features from upsampled features from connected upsampling modules, to obtain second multi-layer convolutional extracted features; at least one feature merger is used to perform addition operations on the second multi-layer convolutional extracted features from connected second-type multi-layer convolutional modules and the convolutional features from connected convolutional modules to obtain merged features; at least one upsampling module is used to upsample the merged features from connected feature mergers to obtain upsampled features; wherein, the plurality of target extracted features include the second multi-layer convolutional extracted features output by the second multi-layer convolutional module connected to the first multi-layer convolutional module and the upsampled features output by each of the upsampling modules.

[0136] In one possible implementation, the plurality of sequentially connected first-type multi-layer convolutional modules include a first multi-layer convolutional module, a second multi-layer convolutional module, a third multi-layer convolutional module, and a fourth multi-layer convolutional module; the at least one convolutional module includes a first convolutional module, a second convolutional module, and a third convolutional module; the at least one second-type multi-layer convolutional module includes a fifth multi-layer convolutional module, a sixth multi-layer convolutional module, and a seventh multi-layer convolutional module; the at least one feature merger includes a first feature merger, a second feature merger, and a third feature merger; and the at least one upsampling module includes a first upsampling module, a second upsampling module, and a third upsampling module. A convolutional module is used to extract features from the forgery probability of each video frame to obtain a first extracted feature; the first convolutional module is used to perform a one-dimensional convolution operation on the first extracted feature to obtain a first convolutional feature; a second multi-layer convolutional module is used to extract features from the first extracted feature to obtain a second extracted feature; the second convolutional module is used to perform a one-dimensional convolution operation on the second extracted feature to obtain a second convolutional feature; a third multi-layer convolutional module is used to extract features from the second extracted feature to obtain a third extracted feature; the third convolutional module is used to perform a one-dimensional convolution operation on the third extracted feature to obtain a third convolutional feature. The fourth multi-layer convolutional module is used to extract features from the third extracted features to obtain a fourth extracted feature; the fifth multi-layer convolutional module is used to extract features from the fourth extracted features to obtain a fifth extracted feature; the first feature merger is used to perform an addition operation on the fifth extracted feature and the third convolutional feature to obtain a first merged feature; the first upsampling module is used to upsample the first merged feature to obtain a first upsampled feature; the sixth multi-layer convolutional module is used to extract features from the first upsampled feature to obtain a sixth extracted feature; the second feature merger is used to combine the sixth extracted feature and the second convolutional feature... The first feature extraction module performs an addition operation on the second merged feature to obtain a second merged feature; the second upsampling module upsamples the second merged feature to obtain a second upsampled feature; the seventh multi-layer convolution module extracts features from the second upsampled feature to obtain a seventh extracted feature; the third feature merger performs an addition operation on the seventh extracted feature and the first convolutional feature to obtain a third merged feature; the third upsampling module upsamples the third merged feature to obtain a third upsampled feature; wherein, the plurality of target extracted features include the fifth extracted feature, the first upsampled feature, the second upsampled feature, and the third upsampled feature.

[0137] In one possible implementation, the combination module includes: an eighth multi-layer convolutional module for extracting features based on the fused features to obtain an eighth extracted feature; a fourth convolutional module for performing a one-dimensional convolution operation on the eighth extracted feature to obtain the time information of each forged segment in the video to be detected, wherein the parameters of the fourth convolutional module are set for the time information output scenario; and a fifth convolutional module for performing a one-dimensional convolution operation on the eighth extracted feature to obtain the forgery probability of each forged segment in the video to be detected, wherein the parameters of the fifth convolutional module are set for the forgery probability output scenario.

[0138] In one possible implementation, the device further includes a training module for: acquiring a training sample set, the training sample set including multiple samples, each sample including a sample video and a sample video tag corresponding to the sample video, the sample video tag being used to indicate the authenticity of the corresponding sample video; training an initial model using the training sample set until the loss function value of the model meets a preset condition to obtain a time-localization model, the loss function used for training including a one-dimensional distance intersection-union function.

[0139] In one possible implementation, the apparatus further includes a processing module for: if it is determined that the time information of multiple forged segments overlaps in time, then deleting and / or merging the time information of the multiple forged segments based on the forgery probability of each forged segment.

[0140] In one possible implementation, the device further includes a selection module for: selecting, from all forged segments, a forged segment with a forged probability greater than a preset threshold as the target result for the video to be detected.

[0141] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0142] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0143] This disclosure also provides a non-volatile computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0144] This disclosure also provides a computer program product, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above method.

[0145] Figure 6 A block diagram of a video timing device provided in an embodiment of this disclosure is shown. For example, device 1900 may be provided as a server or terminal device. (Refer to...) Figure 6 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0146] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output interface 1958 (I / O interface). Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0147] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of the device 1900 to perform the above-described method.

[0148] Computer-readable storage media can be tangible devices capable of holding and storing programs / instructions used by instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0149] The computer program (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage medium in the respective computing / processing device.

[0150] The computer program (or computer program instructions) used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions to implement various aspects of this disclosure.

[0151] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0152] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0153] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0154] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0155] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A video temporal positioning method, characterized in that, The method includes: The authenticity of the video to be tested is detected, and the detection results for each video frame in the video to be tested are obtained. The detection results represent the probability of the video frame being forged. The forgery probability of each video frame is input into the time localization model for calculation to obtain the time information of each forged segment of the video to be detected and the forgery probability of each forged segment. The time-localization model includes: a feature extraction module, used to perform multiple feature extractions and multiple upsamplings based on the forgery probability of each video frame to obtain multiple target extracted features; an attention layer, used to perform feature fusion based on the multiple target extracted features to obtain fused features; and a combination module, used to determine the time information of each forged segment and the forgery probability of each forged segment based on the fused features. The method trains the temporal localization model using a one-dimensional adaptive training sample selection strategy. During training, for each real forged segment corresponding to a target box, multiple anchor boxes closest to the center are selected as candidate positive samples. The Intersection over Union (IoU) values ​​between the multiple candidate anchor boxes and the target box are calculated, thereby calculating the mean and standard deviation. The sum of the mean and the standard deviation is set as the IoU threshold. Anchor boxes with IoU values ​​greater than or equal to the IoU threshold and whose centers are within the target box are selected as positive samples.

2. The method according to claim 1, characterized in that, The authenticity of the video to be tested is verified, and the detection results for each video frame in the video to be tested are obtained, including: Extract each target frame pair and a first audio feature for each target frame pair from the video to be detected. Each target frame pair includes two adjacent video frames. The first audio feature includes Mel spectrum information. Each target frame pair and a first audio feature for that target frame pair are input into a forgery detection model for calculation to obtain a score for each target frame pair, wherein the score represents the forgery probability of the target frame pair; The detection result for each video frame in the video to be detected is determined based on the score of each target frame pair.

3. The method according to claim 1, characterized in that, The feature extraction module includes: Multiple first-type multi-layer convolutional modules are connected in sequence. Each first-type multi-layer convolutional module is used to extract features from the forgery probability of each video frame, or to extract features from the first multi-layer convolutional extracted features from the connected previous first-type multi-layer convolutional module, to obtain the first multi-layer convolutional extracted features. At least one convolutional module, each of the convolutional modules being used to perform a one-dimensional convolution operation on the first multi-layer convolutional features extracted from the first multi-layer convolutional features of the first type of multi-layer convolutional modules connected in a one-dimensional convolutional manner to obtain convolutional features; At least one second-type multi-layer convolutional module, each second-type multi-layer convolutional module is used to extract features from the first multi-layer convolutional features extracted from the connected first-type multi-layer convolutional modules, or to extract features from the upsampled features extracted from the connected upsampled modules, to obtain the second multi-layer convolutional features. At least one feature merger, each of the feature mergers being used to perform an addition operation on the second multi-layer convolutional features extracted from the second type of multi-layer convolutional modules and the convolutional features from the connected convolutional modules to obtain merged features; At least one upsampling module, each of the upsampling modules being used to upsample the merged features from the connected feature mergers to obtain upsampled features; The multiple target extraction features include the second multi-layer convolution extraction features output by the second multi-layer convolution module connected to the first multi-layer convolution module, and the upsampling features output by each of the upsampling modules.

4. The method according to claim 3, characterized in that, The plurality of sequentially connected first-type multi-layer convolutional modules include a first multi-layer convolutional module, a second multi-layer convolutional module, a third multi-layer convolutional module, and a fourth multi-layer convolutional module; the at least one convolutional module includes a first convolutional module, a second convolutional module, and a third convolutional module; the at least one second-type multi-layer convolutional module includes a fifth multi-layer convolutional module, a sixth multi-layer convolutional module, and a seventh multi-layer convolutional module; the at least one feature merger includes a first feature merger, a second feature merger, and a third feature merger; and the at least one upsampling module includes a first upsampling module, a second upsampling module, and a third upsampling module. The first multi-layer convolutional module is used to extract features from the forgery probability of each video frame to obtain the first extracted features; The first convolution module is used to perform a one-dimensional convolution operation on the first extracted features to obtain the first convolution feature; The second multi-layer convolutional module is used to extract features from the first extracted features to obtain the second extracted features; The second convolution module is used to perform a one-dimensional convolution operation on the second extracted features to obtain the second convolution features; The third multi-layer convolutional module is used to extract features from the second extracted features to obtain the third extracted features; The third convolution module is used to perform a one-dimensional convolution operation on the third extracted features to obtain the third convolution features; The fourth multi-layer convolutional module is used to extract features from the third extracted features to obtain the fourth extracted features; The fifth multi-layer convolutional module is used to extract features from the fourth extracted features to obtain the fifth extracted features; The first feature merger is used to perform an addition operation on the fifth extracted feature and the third convolutional feature to obtain the first merged feature; The first upsampling module is used to upsample the first merged feature to obtain the first upsampled feature; The sixth multi-layer convolutional module is used to extract features from the first upsampled features to obtain the sixth extracted features; The second feature merger is used to perform an addition operation on the sixth extracted feature and the second convolutional feature to obtain the second merged feature; The second upsampling module is used to upsample the second merged feature to obtain the second upsampled feature; The seventh multi-layer convolutional module is used to extract features from the second upsampled features to obtain the seventh extracted features; The third feature merger is used to perform an addition operation on the seventh extracted feature and the first convolutional feature to obtain the third merged feature; The third upsampling module is used to upsample the third merged feature to obtain the third upsampled feature; The plurality of target extraction features include the fifth extraction feature, the first upsampling feature, the second upsampling feature, and the third upsampling feature.

5. The method according to any one of claims 1, 3, and 4, characterized in that, The combined module includes: The eighth multi-layer convolutional module is used to extract features based on the fused features to obtain the eighth extracted features; The fourth convolution module is used to perform one-dimensional convolution operations on the eighth extracted features to obtain the time information of each forged segment in the video to be detected. The parameters of the fourth convolution module are set for the time information output scenario. The fifth convolution module is used to perform one-dimensional convolution operations on the eighth extracted features to obtain the forgery probability of each forged segment in the video to be detected. The parameters of the fifth convolution module are set for the forgery probability output scenario.

6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Obtain a training sample set, which includes multiple samples. Each sample includes a sample video and a sample video tag corresponding to the sample video. The sample video tag is used to indicate the authenticity of the corresponding sample video. The initial model is trained using the training sample set until the loss function value of the model meets the preset conditions, thus obtaining the time-localization model. The loss function used for training includes the one-dimensional distance intersection-union function.

7. The method according to any one of claims 1 to 4, characterized in that, The method further includes: If it is determined that the time information of multiple forged segments overlaps, the time information of the multiple forged segments is reduced and / or merged based on the forgery probability of each forged segment.

8. The method according to any one of claims 1 to 4, characterized in that, The method further includes: From all forged segments, the forged segments with a forgery probability greater than a preset threshold are selected as the target result for the video to be detected.

9. A video time positioning device, characterized in that, The device includes: The detection module is used to detect the authenticity of the video to be detected and obtain the detection results for each video frame in the video to be detected. The detection results represent the probability of the video frame being forged. The calculation module is used to input the forgery probability of each video frame into the time localization model for calculation, so as to obtain the time information of each forged segment of the video to be detected and the forgery probability of each forged segment. The time-localization model includes: a feature extraction module, used to perform multiple feature extractions and multiple upsamplings based on the forgery probability of each video frame to obtain multiple target extracted features; an attention layer, used to perform feature fusion based on the multiple target extracted features to obtain fused features; and a combination module, used to determine the time information of each forged segment and the forgery probability of each forged segment based on the fused features. The device trains the temporal localization model using a one-dimensional adaptive training sample selection strategy. During training, for each real forged segment corresponding to a target box, multiple anchor boxes closest to the center are selected as candidate positive samples. The Intersection over Union (IoU) values ​​between the multiple candidate anchor boxes and the target box are calculated, thereby calculating the mean and standard deviation. The sum of the mean and the standard deviation is set as the IoU threshold. Anchor boxes with IoU values ​​greater than or equal to the IoU threshold and whose centers are within the target box are selected as positive samples.

Images