Video detection model training method and device, electronic equipment and storage medium

Abstract

The invention provides a video detection model training method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a real video sample carrying annotation information; editing at least one frame of image in the real video sample to obtain a false video sample; determining a difference between the real video sample and the false video sample, and determining annotation information of the false video sample based on the difference; and updating parameters of the video detection model based on the real video sample, the false video sample and the corresponding annotation information to obtain a trained video detection model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a video detection model training method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Computer vision (CV) is a science that studies how to make machines "see". More specifically, it refers to using cameras and computers to replace human eyes to identify, track, and measure targets, and further perform image processing to make computer processing more suitable for human eye observation or transmission to instrument detection. Video detection model is an important technology in the field of computer vision, mainly used for analyzing and identifying specific information in video content, such as face authenticity, scene change, behavior, etc.

[0003] Among the many application scenarios of video detection model, face authenticity identification technology can help determine whether the face in the video is synthesized by an artificial intelligence model or has been tampered with. In related technologies, a video detection model for face authenticity identification is supervised and trained based on a binary classification label of real and fake, which makes the model unable to learn more fine-grained information in the sample. SUMMARY

[0004] The embodiments of the present application provide a video detection model training method and device, an electronic device, and a storage medium, which can enable the video detection model to learn the differential features between real and fake video samples.

[0005] The technical solutions of the embodiments of the present application are as follows:

[0006] The embodiments of the present application provide a video detection model training method, which comprises:

[0007] Obtaining a real video sample carrying label information, wherein the real video sample comprises multiple frames of images;

[0008] Performing editing processing on at least one frame of image in the real video sample to obtain a fake video sample;

[0009] Determining the difference between the real video sample and the fake video sample, and determining the label information of the fake video sample based on the difference, wherein the label information of the fake video sample is used to represent the editing trace of the fake video sample;

[0010] Updating the parameters of the video detection model based on the real video sample, the fake video sample, and the corresponding label information respectively to obtain a trained video detection model.

[0011] The embodiment of the application provides a training device of a video detection model, and the device comprises:

[0012] A sample construction module is configured to acquire a real video sample carrying label information, wherein the real video sample comprises multiple images;

[0013] The sample construction module is further configured to perform editing processing on at least one image in the real video sample to obtain a false video sample;

[0014] The sample construction module is further configured to determine a difference between the real video sample and the false video sample, and determine label information of the false video sample based on the difference, wherein the label information of the false video sample is used to represent an editing trace of the false video sample;

[0015] A parameter updating module is configured to update parameters of the video detection model based on the real video sample, the false video sample and respective corresponding label information, to obtain a trained video detection model.

[0016] The embodiment of the application provides an electronic device, comprising:

[0017] A memory is configured to store computer executable instructions;

[0018] A processor is configured to execute the computer executable instructions stored in the memory, and implement the training method of the video detection model provided by the embodiment of the application.

[0019] The embodiment of the application provides a computer readable storage medium, which stores a computer program or computer executable instructions, and is used to implement the training method of the video detection model provided by the embodiment of the application when executed by a processor.

[0020] The embodiment of the application provides a computer program product, comprising a computer program or computer executable instructions, and the computer program or computer executable instructions are executed by a processor to implement the training method of the video detection model provided by the embodiment of the application.

[0021] The embodiment of the application has the following beneficial effects:

[0022] Based on the difference between the real video sample and the false video sample, the label information of the false video sample is accurately determined, the editing trace of the false video sample is represented by the label information, instead of a simple binary classification result, and then the parameters of the video detection model are updated based on the real video sample, the false video sample and respective corresponding label information, so that the video detection model can learn the differential features between the real and false video samples, thereby significantly enhancing the recognition ability of the model to the false video. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 FIG. 1 is a structural schematic diagram of a training system architecture of a video detection model provided by an embodiment of the present application;

[0024] Figure 2 FIG. 2 is a structural schematic diagram of an electronic device for training of a video detection model provided by an embodiment of the present application;

[0025] Figure 3A FIG. 3 is a first flow schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0026] Figure 3B FIG. 4 is a second flow schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0027] Figure 3C FIG. 5 is a third flow schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0028] Figure 3D FIG. 6 is a fourth flow schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0029] Figure 3E FIG. 7 is a fifth flow schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0030] Figure 3F FIG. 8 is a sixth flow schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0031] Figure 3G FIG. 9 is a seventh flow schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0032] Figure 3H FIG. 10 is an eighth flow schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0033] Figure 4 FIG. 11 is a first application schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0034] Figure 5 FIG. 12 is a second application schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0035] Figure 6 FIG. 13 is a first principle schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0036] Figure 7 FIG. 14 is a second principle schematic diagram of a training method of a video detection model provided by an embodiment of the present application;

[0037] Figure 8 FIG. 3 is a third principle schematic diagram of a training method of a video detection model according to an embodiment of the present application;

[0038] Figure 9 FIG. 4 is a fourth principle schematic diagram of a training method of a video detection model according to an embodiment of the present application;

[0039] Figure 10 FIG. 5 is a fifth principle schematic diagram of a training method of a video detection model according to an embodiment of the present application.

[0040] It should be noted that the above-mentioned "first", "second" are only used to distinguish different schemes, and do not represent the advantages or disadvantages of the schemes or the priority in the implementation process. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be described in further detail below with reference to the drawings, and the described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0042] In the following description, "some embodiments" are referred to, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict.

[0043] In the following description, the terms "first\second\third" are only used to distinguish similar objects, and do not represent a specific order of the objects, and it can be understood that "first\second\third" can interchange the specific order or sequence as allowed, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0044] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as processing circuitry or memory) or a combination thereof. Similarly, one processor (or multiple processors or memory) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the functions of the module or unit.

[0045] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0046] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0047] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0048] 1) Video Editing Detection: Video editing detection is a technology used to identify whether video content has been edited or modified. Video editing is a technology that involves modifying and creating video content, and can include various operations such as cutting, splicing, color correction, and adding special effects. Among these, editing facial videos through video face-swapping technology is an advanced form of video editing. It usually uses artificial intelligence algorithms, such as deep learning networks, to replace facial images in the video and achieve real-time facial transformation.

[0049] 2) Video Diffusion Model: A deep learning model that generates video content based on a diffusion process, typically used in applications such as video synthesis, editing, and enhancement. The basic idea of ​​the video diffusion model is to decompose video generation into two processes: a forward diffusion process and a backward denoising process. In the forward diffusion process, the model gradually converts a complete video frame into random noise. In the backward denoising process, the model attempts to recover the original video frame from the noise.

[0050] 3) Sample: In machine learning and deep learning, a sample refers to a single data instance used to train a model. Each sample typically contains input data and corresponding annotation information to guide the model's learning process. Samples can be various forms of data, such as images, text, audio, or video frames.

[0051] 4) Label information: This refers to the target output data associated with each sample. It provides the model with the learning target. Label information can take many forms, depending on the type of task being performed. For example, in classification tasks, label information is usually a discrete category label, such as "cat", "dog" or "car". In regression tasks, label information is a continuous numerical value, such as age, temperature, etc.

[0052] 5) Binary classification labels: These are data labels used to indicate that a sample belongs to one of two categories. This type of label is a basic component of binary classification problems, where each sample is labeled as "yes" or "no", "positive" or "negative", "0" or "1", etc., to indicate that the sample belongs to a specific category.

[0053] 6) Triangular mesh: Triangular mesh is a basic concept in computer graphics and geometric modeling. It is a 3D model composed of multiple triangular faces. These triangular faces are interconnected by sharing vertices to form a connected structure. Each triangle consists of three vertices, and each vertex has its corresponding three-dimensional coordinates.

[0054] This application provides a training method, apparatus, electronic device, and computer-readable storage medium for a video detection model, which enables the video detection model to learn the differential features between real video samples and fake video samples.

[0055] The training method for the video detection model provided in this application embodiment can be implemented by the terminal or the server alone, or by the terminal and the server working together. For example, the terminal can undertake the training method for the video detection model described below alone, or the terminal can send a training request for the video detection model containing real video samples to the server. The server, based on the received training request for the video detection model containing real video samples, executes the training method for the video detection model, edits at least one frame of the real video sample to obtain a fake video sample, determines the annotation information of the fake video sample based on the difference between the real video sample and the fake video sample, and updates the parameters of the video detection model based on the real video sample, the fake video sample, and the corresponding annotation information to obtain the trained video detection model.

[0056] See Figure 1 , Figure 1 This is a schematic diagram of the structure of the training system architecture of the video detection model provided in this application embodiment. In the training system 10 of the video detection model provided in this application embodiment, in order to support the training application of a video detection model, the terminal 400 connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0057] Terminal 400 can be used to obtain training requests for video detection models containing real video samples. For example, if a user inputs real video samples through terminal 400, terminal 400 will obtain the real video samples and automatically generate training requests for video detection models containing the real video samples.

[0058] In some embodiments, a training plugin for a video detection model can be embedded in the client running in the terminal 400. This plugin is used to execute the training method of the video detection model locally on the client. For example, after the terminal 400 obtains a training request for a video detection model containing real video samples, it calls the training plugin for the video detection model to edit at least one frame of the real video samples to obtain fake video samples. Based on the differences between the real video samples and the fake video samples, it determines the annotation information of the fake video samples. Based on the real video samples, the fake video samples, and their corresponding annotation information, it updates the parameters of the video detection model to obtain the trained video detection model.

[0059] It is worth noting that the trained video detection model can be stored locally on the terminal 400 and retrieved when needed.

[0060] In some embodiments, after the terminal 400 obtains a training request for a video detection model containing real video samples, it calls the training interface of the video detection model of the server 200 through the network 300. The server 200 executes the training method of the video detection model according to the received training request for the video detection model containing real video samples, edits at least one frame of the real video samples to obtain fake video samples, determines the annotation information of the fake video samples based on the differences between the real video samples and the fake video samples, and updates the parameters of the video detection model based on the real video samples, the fake video samples and their corresponding annotation information to obtain the trained video detection model.

[0061] It is worth noting that the trained video detection model can be stored locally on server 200 and retrieved when needed.

[0062] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0063] Terminal 400 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, smart voice interaction device, smart home appliance, vehicle terminal, aircraft, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0064] See Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device for training a video detection model provided in an embodiment of this application. Figure 2 The electronic device 500 shown can be Figure 1 The terminal 400 or server 200 in the electronic device 500 includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in the electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 540.

[0065] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0066] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices physically located away from the processor 510.

[0067] The memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0068] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0069] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0070] The network communication module 552 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0071] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A training device 555 for a video detection model stored in memory 550 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: a sample construction module 5551 and a parameter update module 5552. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0072] As mentioned above, the electronic device implementing the training method of the video detection model in this application can be a terminal, a server, or a combination of both. Therefore, the executing entity of each step will not be described again below. See [link to relevant documentation]. Figure 3A , Figure 3A This is a schematic diagram of the first process of training the video detection model provided in this application embodiment, which will be combined with Figure 3A The steps shown are explained.

[0073] In step 101, real video samples carrying annotation information are obtained.

[0074] As an example, a real video sample consists of multiple consecutive images. A real video sample is a video sample that has not been edited, meaning that each frame in a real video sample has not been edited. A consecutive image refers to a sequence of images that have a smooth transition in time and space between adjacent frames.

[0075] In step 102, at least one frame of the real video sample is edited to obtain a fake video sample.

[0076] As an example, a fake video sample refers to an edited video sample. A fake video sample is characterized by including at least one edited image. At least one image is selected from multiple consecutive images in a real video sample for editing to obtain at least one edited image. The selected image in the real video sample is replaced with the edited image to obtain a fake video sample. The process of selecting at least one image from multiple consecutive images in a real video sample for editing can be as follows: randomly selecting at least one image to be edited from multiple consecutive images in a real video sample for editing; selecting all images in the real video sample for editing; or selecting a target image from multiple consecutive images in a real video sample for editing.

[0077] In some embodiments, see Figure 3B , Figure 3B This is a schematic diagram of the second process of the training method for the video detection model provided in the embodiments of this application. Figure 3A Step 102 shown can be implemented through the following steps 1021 to 1022, which are explained in detail below.

[0078] In step 1021, target recognition processing is performed on the real video sample to obtain at least one frame of target image.

[0079] As an example, target recognition processing is performed on real video samples to obtain at least one image frame that meets the target recognition conditions. The image that meets the target recognition conditions and the images adjacent to the image that meets the target recognition conditions are used together as the target image. Here, the image adjacent to the image that meets the target recognition conditions refers to the image frame that is closely connected to the image that meets the target recognition conditions in the time sequence of the video sample. The target recognition conditions refer to the set behavior or set expression of the target. For example, in the field of facial authenticity recognition, the target recognition conditions may include the appearance of facial actions such as blinking, opening the mouth, and bowing the head. The target recognition conditions may also include the appearance of facial expressions such as smiling and surprise. There is no limitation here. When the target recognition condition is blinking, target recognition processing is performed on real video samples. When blinking is detected in a certain frame of the real video sample, the image frame of blinking is obtained. The five frames before and the five frames after the image frame of blinking are obtained. The image frame of blinking and the five frames before and after the image frame of blinking are used as the target image.

[0080] In the embodiments of this application, the method of combining an image that meets the target recognition conditions with its adjacent image frames as the target image can more accurately identify and locate the occurrence of target recognition conditions in the video, ensuring the continuity of target recognition conditions. This not only effectively avoids the target recognition interruption or error that may occur due to relying on only a single frame, but also reduces the situation of false detection and missed detection.

[0081] In step 1022, at least one frame of the target image is edited to obtain a fake video sample.

[0082] As an example, at least one frame of the target image is edited to obtain an edited image corresponding to each target image. The target image in the real video sample is replaced with the corresponding edited image, and the replaced real video sample is used as a fake video sample.

[0083] In some embodiments, see Figure 3C , Figure 3C This is a schematic diagram of the third process of the training method for the video detection model provided in the embodiments of this application. Figure 3B Step 1022 shown can be achieved by performing the following steps 201 to 204 for each frame of the target image, as explained in detail below.

[0084] In step 201, the first feature point of the target image is extracted, and the feature value of the first feature point is adjusted. The feature point corresponding to the adjusted feature value is determined as the second feature point.

[0085] As an example, based on the feature point extraction algorithm, the first feature point of the target image is extracted, and the feature value of the first feature point is randomly increased or decreased, and the adjusted feature point is determined as the second feature point.

[0086] For example, in the field of facial recognition, the target image includes facial information. The first feature point of the facial information in the target image is extracted by the Face Alignment algorithm. The first feature point includes 68 facial feature points, including feature points of the two eyes, nose, mouth and cheeks. The feature value of the facial feature point is the position information (x, y) of the facial feature point. The position information of the facial feature point is randomly increased or decreased to obtain the adjusted position information (x+6, y-5). The adjusted 68 facial feature points are determined as the second feature point.

[0087] In step 202, the first feature point and the second feature point are triangulated to obtain the first mesh and the second mesh.

[0088] As an example, the first feature point is triangulated into multiple triangles, so that each feature point can be included in one or more triangular meshes, thus obtaining the first mesh. The second feature point is triangulated to obtain the second mesh, wherein the mesh includes triangular meshes.

[0089] In step 203, the target image is transformed based on the transformation relationship between the first grid and the second grid to obtain the edited image.

[0090] As an example, based on the transformation relationship between the first grid and the second grid, a transformation relationship matrix is ​​calculated. Based on the transformation relationship matrix, the position information of the pixels included in the target image is transformed to obtain the edited position information. Based on the edited position information of the pixels included in the target image, the edited image is drawn. The transformation relationship may include affine transformation and projection transformation.

[0091] In step 204, the target image in the real video sample is replaced with the edited image to obtain a fake video sample.

[0092] As an example, the images in the real video sample, except for the target image, remain unchanged. Only the target image in the real video sample is replaced with the corresponding edited image to obtain a fake video sample. The fake video sample includes the edited image and the images in the real video sample, except for the target image.

[0093] For example, a real video sample includes target image 1, target image 2, and target graphic 3. Target image 1, target image 2, and target graphic 3 are edited to obtain edited images of target image 1, target image 2, and target graphic 3, respectively. Target image 1 in the real video sample is replaced with the edited image of target image 1, target image 2 is replaced with the edited image of target image 2, and target image 3 is replaced with the edited image of target image 3. The replaced real video sample is then used as a fake video sample.

[0094] In this embodiment of the application, by editing real video samples to generate fake video samples, the model can not only learn the difference features between real and fake video samples more effectively, but also generate a large amount of training data, effectively improving the model's adaptability to diversity and variability.

[0095] In step 103, the differences between real video samples and fake video samples are determined, and the annotation information of fake video samples is determined based on the differences.

[0096] As an example, by comparing the differences between real video samples and fake video samples frame by frame, the annotation information of fake video samples is determined. The annotation information of fake video samples includes edited video and edited text. Edited video is used to indicate the video frames with edited traces and the location of the edited traces in the video frames with edited traces. Edited text is the text used to indicate the edited traces, such as "edited traces when the human blinks".

[0097] In some embodiments, see Figure 3D , Figure 3D This is a schematic diagram of the fourth step of the training method for the video detection model provided in this application embodiment. Figure 3A The step 103 shown, “determining the difference between real video samples and fake video samples”, can be achieved through the following steps 1031 to 1033, which are explained in detail below.

[0098] In step 1031, the real video sample includes multiple first images, and from the fake video sample, the second image corresponding to each first image is determined.

[0099] As an example, both the real video sample and the fake video sample include multiple frames of images. The first image is any frame of the real video sample, and the second image is the image corresponding to the first image among the multiple frames of the fake video sample. The second image is obtained by editing the first image. The frame number of the second image corresponding to the first image in the fake video sample is the same as the frame number of the first image in the real video sample. The specific steps of the editing process can be found in step 1022 above, and will not be repeated here.

[0100] In step 1032, the difference information between each first image and the corresponding second image is determined.

[0101] As an example, the first image includes multiple first pixels, and the second image includes multiple second pixels. The difference information between the first image and the corresponding second image is determined by comparing whether the first pixels of the first image and the second pixels of the second image at the same position are consistent.

[0102] In some embodiments, Figure 3DStep 1032 shown can be implemented by performing the following processing on any position in the first image: determining a blank image, and performing the following processing on each position in the blank image: determining a first pixel at the position in the first image, and determining a second pixel at the position in the second image; when there is a difference between the first pixel and the second pixel, assigning the pixel at the position in the blank image to a first set pixel; when there is no difference between the first pixel and the second pixel, assigning the pixel at the position in the blank image to a second set pixel; and determining the blank image after assignment as difference information.

[0103] As an example, the size of the blank image is the same as the size of the first image. The pixel value of each pixel in the blank image is 0 or empty. The set pixel refers to the pixel that is set to mark the position of whether there are editing traces in the image. The first set pixel is used to mark the position of editing traces in the image, and the second set pixel is used to mark the position of no editing traces in the image. When there is a difference between the first pixel and the second pixel, it means that there are editing traces in the first image and the second image at the current position. Then the pixel at the corresponding position in the blank image is assigned the value of the first set pixel. When there is no difference between the first pixel and the second pixel, it means that there are no editing traces in the first image and the second image at the current position. The pixel at the corresponding position in the blank image is assigned the value of the second set pixel. The blank image after assignment is determined as the difference information.

[0104] For example, see Figure 7 , Figure 7 This is a schematic diagram of the second principle of the training method of the video detection model provided in this application embodiment. The edit trace video 703 includes a blank image after assignment. The black part of the blank image after assignment is the area without edit traces. The pixels in the area without edit traces are the second set pixels (black pixels). The bright part of the blank image after assignment is the area with edit traces. The pixels in the area with edit traces are the first set pixels that are set as highlight pixels.

[0105] In step 1033, multiple difference information is identified as the differences between real video samples and fake video samples.

[0106] In some embodiments, the annotation information of fake video samples is used to characterize editing traces in the virtual video samples. Figure 3AThe step 103 shown, "determining the annotation information of fake video samples based on differences", can be achieved through the following steps: when the annotation information of a fake video sample includes edit trace video, multiple blank images with assigned values ​​corresponding to multiple difference information are combined into an edit trace video; when the annotation information of a fake video sample includes edit trace text, the edit trace video and the real video are overlaid to obtain a visualized edit trace video; and the visualized edit trace video is subjected to text analysis processing to obtain the edit trace text.

[0107] As an example, when the annotation information of a fake video sample includes edited video, multiple assigned blank images are synthesized into an edited video in chronological order. When the annotation information of a fake video sample includes edited text, the visualized edited video is video encoded to obtain its encoded features. Based on a large language model, the encoded features of the visualized edited video are analyzed to obtain the edited text. Alternatively, the edited text corresponding to the visualized edited video can be summarized through manual analysis.

[0108] For example, see Figure 7 ,Will Figure 7 The edit trace video 703 is overlaid with the real video 701 to obtain the visualized edit trace video 704. Users can more intuitively observe the areas of the fake video sample that have been edited compared to the real video sample, and then construct the corresponding edit trace text information.

[0109] In this embodiment, the annotation information of fake video samples represents the difference between real video samples and fake video samples. Compared with related technologies, it reduces the dependence on binary classification annotation information and provides richer supervision information (annotation information), making the model more focused on learning the difference features between real video samples and fake video samples.

[0110] In step 104, the parameters of the video detection model are updated based on real video samples, fake video samples, and their corresponding annotation information to obtain the trained video detection model.

[0111] As an example, the annotation information for real video samples and the annotation information for fake video samples can be edit trace text, or the annotation information for real video samples and the annotation information for fake video samples can be edit trace videos. In this case, the edit trace text for real videos includes the sample type, and the edit trace text for fake video samples includes both the sample type and the edit trace information.

[0112] For example, the annotation information for a real video sample is the edited text "This is an unedited video," while the annotation information for a fake video sample is the edited text "This is an edited video, and the edited information is: in the video frame where the person blinks, the person's nose and mouth shrink."; or, the annotation information for a real video sample is a blank edited video where all image frames have null pixels, while the annotation information for a fake video sample is an edited video that includes at least one edited image frame.

[0113] In some embodiments, see Figure 3E , Figure 3E This is a schematic diagram of the fifth step in the training method of the video detection model provided in this application embodiment. Figure 3A Step 104 shown can be implemented through the following steps 1041 to 1043, which are explained in detail below.

[0114] In step 1041, when the video detection model includes a pre-trained model, the parameters of the pre-trained model are updated based on real video samples, fake video samples, and their corresponding annotation information to obtain the initially trained video detection model.

[0115] In some embodiments, see Figure 3F , Figure 3F This is a schematic diagram of the sixth step in the training method of the video detection model provided in this application embodiment. Figure 3E The step 1041 shown, "updating the parameters of the pre-trained model based on real video samples, fake video samples and their corresponding annotation information to obtain the initially trained video detection model", can be achieved through the following steps 301 to 304, which are explained in detail below.

[0116] In step 301, each sample in the sample set is subjected to noise addition to obtain a noisy sample.

[0117] As an example, the sample set includes real video samples and fake video samples. The samples are encoded to obtain the encoded features of the first image sample. The noise diffusion step number is obtained. Based on the noise diffusion step number, the encoded features of the first image sample are denoised. The encoding of the samples can be implemented by a video coding model. Here, there is no restriction on the video coding model. The noise diffusion step number is the number of times the samples are denoised. For example, the noise diffusion step number is a positive integer greater than 1 and less than 1000.

[0118] For example, when the noise diffusion step number is 3, the encoded features of the first image sample are processed with Gaussian noise to obtain the encoded features after the first noise addition. The encoded features after the first noise addition are processed with Gaussian noise to obtain the encoded features after the second noise addition. The encoded features after the second noise addition are processed with Gaussian noise to obtain the sample after noise addition.

[0119] In this embodiment of the application, by introducing noise into the training of the pre-trained model, the model can learn how to effectively handle and predict uncertainties and noise interference that may be encountered in the real world, thereby improving the robustness of the model to environmental changes and noise in practical applications. At the same time, the presence of noise increases the diversity and complexity of the training data, which helps the model to better adapt to new and unseen video samples and improve the model's generalization ability.

[0120] In step 302, the noise-added samples are denoised using a pre-trained model to obtain the editing traces of the samples.

[0121] In some embodiments, the denoising module of the pre-trained model includes M cascaded encoding layers and M cascaded decoding layers, where M is a positive integer greater than 0. See [link to documentation]. Figure 3G , Figure 3G This is a schematic diagram of the seventh step in the training method of the video detection model provided in this application embodiment. Figure 3F Step 302 shown can be implemented through the following steps 3021 to 3023, which are explained in detail below.

[0122] In step 3021, based on the M cascaded coding layers included in the pre-trained model, the noisy samples are encoded to obtain the first coded features.

[0123] As an example, based on the first coding layer, the noisy sample is encoded to obtain the encoded features output by the first coding layer. Based on the m-th coding layer, the encoded features output by the (m-1)-th coding layer are encoded to obtain the encoded features output by the m-th coding layer. Based on the M-th coding layer, the encoded features output by the (M-1)-th coding layer are encoded to obtain the first encoded feature. Here, m is a positive integer greater than 1 and less than M, and the dimension of the encoded features output by the m-th coding layer is less than the dimension of the encoded features output by the (m-1)-th coding layer.

[0124] In some embodiments, the coding layer includes an attention layer, multiple cascaded self-attention layers, and a downsampling layer. The process of "encoding the coding features output by the (m-1)th coding layer based on the m-th coding layer to obtain the coding features output by the m-th coding layer" can be achieved through the following steps: performing attention processing on the coding features output by the (m-1)th coding layer based on a first attention layer to obtain a first attention feature of the m-th coding layer; performing attention processing on the first attention feature based on multiple cascaded second attention layers to obtain a second attention feature of the m-th coding layer; and performing pooling processing on the second attention feature of the m-th coding layer based on the downsampling layer to obtain the coding features output by the m-th coding layer.

[0125] As an example, the first attention layer can be a cross attention layer or a self-attention layer, and the second attention layer is a self-attention layer. Based on the downsampling layer, the second attention features of the m-th encoding layer are pooled to reduce the feature dimension of the second attention features. The pooling of the second attention features of the m-th encoding layer can include max pooling, mean pooling, and random pooling of the second attention features, which are not limited here.

[0126] In some embodiments, attention processing is performed on the input features based on the third attention layer, wherein the query (Q, Query) of the third attention layer is the input feature, and the key (K, Key) and value (V, Value) of the third attention layer are the fused features. The third attention layer is either the first attention layer or the second attention layer, and the fused features are obtained by fusing the encoded features of the sample and the encoded features of the sample's labeled information.

[0127] As an example, the sample is video encoded to obtain the encoded features of the sample. The labeled information of the sample is text encoded or video encoded to obtain the encoded features of the labeled information of the sample. The encoded features of the sample and the encoded features of the labeled information are fused to obtain the fused features. The fused features are used as the keys and values ​​of the third attention layer. The fusion process can be a concatenation of the encoded features of the sample and the encoded features of the labeled information, which is not limited here.

[0128] In step 3022, attention processing is performed on the first encoded feature to obtain the second encoded feature.

[0129] As an example, attention processing is performed on the first encoded feature based on the first attention layer. A domain adaptor is connected before the first attention layer. The first encoded feature is adapted by the domain adaptor to obtain the adapted first encoded feature. Attention processing is then performed on the adapted first encoded feature based on the first attention layer to obtain the second encoded feature.

[0130] In the video detection model of this application embodiment, when performing transfer learning, the first encoded feature is adapted by a domain adaptor, which can reduce the difference between the source domain and the target domain, so that the model trained on the source domain can also have good performance on the target domain. Here, the source domain refers to the domain in which the model training sample set is located, and the target domain refers to the domain outside the domain in which the model training sample set is located.

[0131] In step 3023, based on the M cascaded decoding layers included in the pre-trained model, the second encoded features are decoded to obtain the edit traces of the sample.

[0132] As an example, based on the first decoding layer, the second encoded feature is decoded to obtain the output feature of the first decoding layer. Based on the m-th decoding layer, the output feature of the (m-1)-th decoding layer is decoded to obtain the output feature of the m-th decoding layer. Based on the M-th decoding layer, the output feature of the (M-1)-th decoding layer is decoded to obtain the edit traces of the sample. Here, the dimension of the decoded feature output by the m-th decoding layer is greater than the dimension of the decoded feature output by the (m-1)-th decoding layer.

[0133] In some embodiments, the decoding layer includes an upsampling layer, multiple cascaded second attention layers, and a first attention layer. The process of "decoding the output features of the (m-1)th decoding layer based on the m-th decoding layer to obtain the output features of the m-th decoding layer" can be achieved through the following steps: performing unpooling processing on the attention features of the m-th decoding layer based on the upsampling layer to obtain the unpooled features of the m-th decoding layer; performing attention processing on the unpooled features of the m-th decoding layer based on the multiple cascaded second attention layers to obtain the third attention features of the m-th decoding layer; and performing attention processing on the third attention features based on the first attention layer to obtain the output features of the m-th decoding layer.

[0134] As an example, decoding is the reverse of encoding, and depooling is the reverse of pooling. Depooling includes reverse max pooling and reverse average pooling, which are not limited here.

[0135] In step 303, a first loss function is constructed based on the editing traces of the sample and the corresponding annotation information of the sample.

[0136] As an example, when the annotation information corresponding to the sample is edit trace text, the annotation information corresponding to the sample is processed by text encoding; or when the annotation information corresponding to the sample is edit trace video, the annotation information corresponding to the sample is processed by video encoding to obtain the encoding features of the annotation information. Based on the encoding features of the annotation information and the edit traces of the sample, a first loss function is constructed. The first loss function can be a cross-entropy loss function, root mean square error loss function, mean square error loss function, log loss function, etc., and is not limited here.

[0137] In step 304, the parameters of the pre-trained model are updated based on the first loss function to obtain the initially trained pre-trained model.

[0138] As an example, by minimizing the first loss function, the parameters of the denoising module included in the pre-trained model are iteratively updated until the pre-trained model meets the training completion condition. The pre-trained model that meets the training completion condition is used as the initial pre-trained model. The method for updating the parameters of the denoising module included in the pre-trained model can be gradient descent or other optimization algorithms, which are not limited here. The training completion condition can be that the number of iterations of the pre-trained model parameters is equal to the preset number of iterations, the loss value of the loss function reaches a preset threshold, or the performance evaluation indicators (such as accuracy, recall, etc.) of the pre-trained model on the validation set reach a set threshold, which are not limited here.

[0139] In step 1042, a classification network is added to the initially trained video detection model to obtain the video detection model to be trained.

[0140] As an example, a classification network is added to the initially trained video detection model. The classification network replaces the M cascaded decoding layers in the video detection model to obtain the video detection model to be trained.

[0141] See Figure 10 Below the M cascaded encoding and attention layers of the pre-trained model, that is, between the layers and the encoding layers, a classification network is connected. When training the classification network, the M cascaded encoding and attention layers of the pre-trained model participate in data processing but do not participate in parameter updates. The M cascaded decoding layers of the pre-trained model are in a frozen state, neither participating in data processing nor in parameter updates.

[0142] In this embodiment, by adding a classification network to the initially trained video detection model, the video detection model can more accurately classify videos and determine whether the input video has editing traces while maintaining the advantages of the original structure. At the same time, during the training process, the encoding and decoding layers do not participate in parameter updates, reducing the demand for computing resources and speeding up the training speed of the model.

[0143] In step 1043, the parameters of the classification network in the video detection model to be trained are updated based on real video samples and fake video samples to obtain the trained video detection model.

[0144] As an example, based on real video samples and fake video samples, the annotation information of real video samples and fake video samples is determined respectively. Based on real video samples and fake video samples and their corresponding annotation information, the parameters of the classification network in the video detection model to be trained are updated to obtain the trained video detection model, where the annotation information is binary classification annotation information.

[0145] In some embodiments, see Figure 3H , Figure 3H This is a schematic diagram of the eighth step in the training method of the video detection model provided in this application embodiment. Figure 3E Step 1043 shown can be implemented through the following steps 501 to 504, which are explained in detail below.

[0146] In step 501, the binary classification labeling information corresponding to each sample in the sample set is determined.

[0147] As an example, the sample set includes real video samples and fake video samples. Based on the sample type, the annotation information of the sample is determined to be either positive or negative. When the sample is a real video sample, the binary classification label information of the sample is positive. When the sample is a fake video sample, the corresponding binary classification label information of the sample is negative. The positive label information is used to annotate real video samples, and the negative label information is used to annotate fake video samples.

[0148] For example, when the sample is a real video sample, the corresponding binary classification label information can be "Real" or "1", and when the sample is a fake video sample, the corresponding binary classification label information can be "Fake" or "0".

[0149] In step 502, the coding features of the samples are determined based on the pre-trained model in the video detection model to be trained.

[0150] As an example, the samples are processed based on M cascaded encoding layers and attention layers in the pre-trained model to obtain the encoded features of the samples.

[0151] In some embodiments, Figure 3HStep 502 shown can be achieved by performing the following steps through the pre-trained model of the video detection model to be trained: encoding the sample through the M cascaded encoding layers included in the pre-trained model to obtain the third encoding feature of the sample; and performing attention processing on the third encoding feature to obtain the encoding feature of the sample.

[0152] As an example, the specific steps for encoding samples using a pre-trained model can be found in step 3021 above, and the specific steps for attention processing of the third encoded feature can be found in step 3022 above, which will not be repeated here.

[0153] In step 503, the encoded features of the samples are classified based on the classification network in the video detection model to be trained, and the classification results of the samples are obtained.

[0154] As an example, the classification network includes multiple cascaded dimensionality reduction layers and soft exponential layers, with the dimensionality reduction layers including linear transformation layers and nonlinear activation layers.

[0155] In some embodiments, Figure 3H Step 503 shown can be achieved by the following steps performed by the classification network in the video detection model to be trained: dimensionality reduction of the encoded features of the sample to obtain the dimensionality reduction features of the sample; mapping of the dimensionality reduction features of the sample to obtain the classification result of the sample.

[0156] As an example, the dimensionality reduction of the encoded features of the sample is processed by multiple cascaded dimensionality reduction layers to obtain the dimensionality reduction features of the sample. The dimensionality reduction features of the sample can be mapped based on the soft exponential function (Softmax function) to transform the dimensionality reduction features of the sample into a one-dimensional edit probability distribution of the sample. Based on the edit probability distribution, the classification result of the sample is obtained.

[0157] For example, the encoded features of the sample are reduced in dimensionality using the first dimensionality reduction layer in a cascaded dimensionality reduction layer to obtain the output features of the first dimensionality reduction layer. The output features of the first dimensionality reduction layer are then reduced in dimensionality using the second dimensionality reduction layer in a cascaded dimensionality reduction layer to obtain the dimensionality reduction features of the sample. Based on the soft exponential function (Softmax function), the dimensionality reduction features of the sample are subjected to soft exponential processing to obtain the editing probability distribution [0.1, 0.9]. This indicates that the probability of the sample being a real video is 0.1 and the probability of the sample being a fake video is 0.9, resulting in the classification result of the sample as a fake video.

[0158] In some embodiments, "dimensionality reduction of the encoded features of a sample through the first dimensionality reduction layer in two cascaded dimensionality reduction layers" can be achieved through the following steps: performing a linear transformation on the encoded features of the sample to obtain the encoded features of the sample after linear transformation; and performing a nonlinear activation on the encoded features of the sample after linear transformation to obtain the output features of the first dimensionality reduction layer.

[0159] As an example, the activation function can be a step function, a sigmoid function, a hyperbolic tangent function, a ReLU function, etc., without limitation.

[0160] In step 504, based on the classification results and binary classification annotation information, the parameters of the classification network of the video detection model to be trained are updated to obtain the trained video detection model.

[0161] As an example, based on the classification results and binary classification annotation information, a second loss function is constructed. By minimizing the second loss function, the parameters of the classification network of the video detection model to be trained are iteratively updated until the video detection model to be trained meets the training completion condition. The video detection model to be trained that meets the training completion condition is then used as the trained video detection model. The method for updating the parameters of the classification network of the video detection model to be trained can use gradient descent or other optimization algorithms, which are not limited here. The training completion condition is the same as that of the pre-trained model in step 304 above, and will not be repeated here. The training completion conditions of the video detection model to be trained and the pre-trained model can be the same or different, which are not limited here.

[0162] The following will illustrate an exemplary application of the training method for the video detection model provided in this application embodiment in a real-world application scenario.

[0163] In the field of facial recognition, AIGC technologies such as video face-swapping are used to edit facial videos. (See also...) Figure 4 , Figure 4 This is a schematic diagram of the first application of the training method of the video detection model provided in this application embodiment. The left image is the original video 401, and the right image is the face-swapped video 402 based on the face in the portrait video 403. In applications that require identity verification, the user's identity information is verified by the video of the user's specified actions recorded by the user. If the above video is edited or tampered with, it will seriously affect the normal use of the application and the security of the user's property.

[0164] In related technologies, video editing detection algorithms rely solely on binary or multi-class labels to guide the model in learning to distinguish between unedited videos (real videos, equivalent to the real videos mentioned above) and edited videos (fake videos, equivalent to the fake videos mentioned above). For example, the labels of samples only have binary classification information of real and fake. The data differences between real and fake samples in the training data are often not only due to construction flaws. The model may learn other non-relevant information such as background in the samples, which will affect the generalization of the model and cause the model to perform poorly in cross-data source tests.

[0165] To address the aforementioned issues, this application proposes a training method for a video detection model. This method constructs video pairs of unedited and edited videos using deformation and fusion algorithms, and then generates a flawed video (equivalent to the edited video mentioned above) by frame-by-frame difference generation. The flawed video indicates the frames and edited areas in the sample where editing is present, allowing technicians to intuitively observe the differences between the unedited and edited videos. A video diffusion model (equivalent to the video detection model mentioned above) learns the mapping from unedited and edited videos to the flawed video, enabling the model to distinguish between real and fake videos.

[0166] The trained video detection model in this embodiment can be applied to facial recognition products. See [link to relevant documentation]. Figure 5 , Figure 5 This is a second application diagram of the training method of the video detection model provided in the embodiments of this application. On the "User Information Authentication" page, the user clicks "Next" and "Start Recognition" in sequence to collect facial information in real time. After the real-time facial detection and shooting is completed, the captured video data will be uploaded to the background. The detection module in the background will determine whether the video is an edited video.

[0167] The training method of the video detection model in this application includes three stages: data construction stage, pre-training stage, and model-classification network training stage. The above three stages will be described below.

[0168] 1) Data Construction Stage

[0169] This application's embodiments construct fake videos during the data construction phase. See also... Figure 6 , Figure 6 This is a schematic diagram of the first principle of the training method of the video detection model provided in this application embodiment, which generates fake videos frame by frame based on unedited videos (real videos).

[0170] Step 1: Obtain real video frames. Select the real video frames that need perturbation processing from the real video (equivalent to the first image above). This can be done by processing all frames in the real video, selecting five frames before and after the image of the person with their eyes closed, or selecting random frames from the real video for processing. Step 2: Extract facial feature points based on the face... The alignment algorithm processes the real video frame to obtain 68 facial feature points. Step 3: Construct randomly perturbed facial feature points by randomly perturbing the 68 facial feature points in the real video frame and adding a random value to each feature point (randomly selected feature point + added random value). Step 4: Triangulation is performed on the facial feature points in the real video frame and the randomly perturbed facial feature points. Step 5: A real facial feature point triangular mesh (equivalent to the mesh mentioned above) is constructed based on the triangulation. Step 6: A perturbed facial feature point triangular mesh is constructed based on the triangulation. Step 7: Affine transformation is performed on each triangle mapping relationship based on the relationship between the real facial feature point triangular mesh and the perturbed facial feature point triangular mesh. Step 8: The perturbed frame is determined based on the affine transformation.

[0171] See Figure 7 The perturbation frame after deformation will be used to replace Figure 7 In the real video 701, the video frames that need to be perturbed are used to reduce the size of the nose and mouth of the human figure. Figure 7 The video in question is a fake video (702 error).

[0172] In this embodiment of the application, an edit trace video is constructed during the data construction stage. Based on real video and fake video, an edit trace video (mark video) is constructed. The formula (1.1) for constructing the edit trace video is as follows:

[0173] f mark =|f real -f fake | (1.1)

[0174] Among them, f mark To edit the trace video, f real and f fake These are the video frames for the corresponding real and fake videos.

[0175] See Figure 7Edit-marked videos indicate video frames in fake videos that have been edited, as well as the location of the edit marks on those frames. In edit-marked videos, the pixel value of a frame without edit marks is 0 (completely black), while the pixel value of a pixel with edit marks is >0.

[0176] Construct the text corresponding to the video samples:

[0177] The text corresponding to the video (equivalent to the annotation information above) indicates whether the video has been edited (whether the video is real or fake). For fake videos, the text also includes the frames and locations where editing traces are found, as shown in Table 1:

[0178] Text information for a real video and text information for a fake video:

[0179] Table 1 Comparison of Video Samples and Text Information

[0180]

[0181] 2) During the model pre-training stage

[0182] The task of the pre-training phase is to learn the correspondence between real videos, fake videos, and videos with editing traces. That is, given a video as input, the pre-trained model indicates whether each pixel in each frame of the video has been edited.

[0183] See Figure 8 , Figure 8 This is a schematic diagram of the third principle of the training method of the video detection model provided in the embodiments of this application. The pre-trained model can be a model with U-Net as the core component, such as the stable video diffusion model (SVD), the animated diffusion model (Animatediff), etc., which are not limited here.

[0184] See Figure 8In the pixel space, the input sample is encoded by the encoding module to obtain the latent vector. In the hidden space, an integer T is randomly sampled from a uniform distribution of 1-1000, called the diffusion step number. Based on the noise-adding module, T-th order Gaussian noise is added to the latent vector to obtain the noisy feature. The noise reduction module (U-Net module) processes the noisy feature and the encoded features of the additional information to obtain the denoised feature. The encoded features of the additional information are obtained by encoding the input sample and the text information of the input sample. In the additional information space, the corresponding encoder encodes the input sample and the text information of the input sample to obtain the encoded features of the additional information, which serve as the K and V inputs of the attention layer of the noise reduction module to assist in the noise reduction process. The denoised feature is decoded to obtain the output.

[0185] The U-Net module in the pre-trained model can be found here. Figure 9 , Figure 9 This is a schematic diagram illustrating the fourth principle of the training method for the video detection model provided in this application embodiment.

[0186] When training a pre-trained model, all parts except the U-Net module of the pre-trained model are frozen and do not participate in training.

[0187] 3) During the classification network training phase

[0188] After the pre-trained model has been trained, all parameters in the pre-trained model are frozen. See [link / reference]. Figure 10 , Figure 10 This is a schematic diagram of the fifth principle of the training method of the video detection model provided in this application embodiment. A classification network (Adapter network) is connected in the middle of the U-Net module in the pre-trained model. During the training phase of the classification network, only the parameters in the classification network are updated. The supervision label of the sample is a binary label, including real label and fake label.

[0189] In summary, the training method for the video detection model provided in this application improves the generalization of the video detection model by providing more supervision information (edited video and text information). On a cross-data source test set, the interception rate of the video detection model in this application can reach 86% (compared to only 12% in related technologies). The U-Net module in the video detection model can better encode temporal and spatial information. By combining an image that meets the target recognition conditions with its neighboring image frames as the target image, the appearance of target recognition conditions in the video can be more accurately identified and located, ensuring the continuity of target recognition conditions. This not only effectively avoids target recognition interruptions or errors that may occur due to relying on only a single frame, but also reduces false detections and missed detections. By editing real video samples to generate fake video samples, the model can not only learn the difference features between real and fake video samples more effectively, but also generate a large amount of training data, effectively improving the model's adaptability to diversity and variability. Overlay processing yields visualized edit traces in the video, allowing users to more intuitively observe the edited areas of fake video samples compared to real video samples, thus constructing corresponding edit trace text information. The annotation information of fake video samples characterizes the differences between real and fake video samples. Compared to related technologies, this reduces reliance on binary classification annotation information and provides richer supervision information (annotation information), enabling the model to focus more on learning the difference features between real and fake video samples. By introducing noise into the training of the pre-trained model, the model can learn how to effectively handle and predict uncertainties and noise interference that may be encountered in the real world, improving the model's robustness to environmental changes and noise in practical applications. Simultaneously, the presence of noise increases the diversity and complexity of training data, helping the model better adapt to new and unseen video samples, improving the model's generalization ability. Adapting the first encoded feature through a domain adaptor can reduce the source domain (Source...) The difference between the source domain and the target domain allows a model trained on the source domain to perform well in the target domain as well. By adding a classification network to the initially trained video detection model, the video detection model can classify videos more accurately and determine whether there are editing traces in the input video while maintaining the advantages of the original structure. At the same time, the encoding and decoding layers do not participate in parameter updates during training, reducing the demand for computing resources and speeding up the training of the model.

[0190] The following description continues to illustrate the exemplary structure of the training device 555 for the video detection model provided in this application embodiment as a software module. In some embodiments, such as... Figure 2As shown, the software modules in the training device 555 for the video detection model stored in the memory 550 may include:

[0191] The sample construction module 5551 is used to acquire real video samples carrying annotation information, wherein the real video samples include multiple frames of images; to edit at least one frame of the real video samples to obtain fake video samples; to determine the differences between the real video samples and the fake video samples, and to determine the annotation information of the fake video samples based on the differences, wherein the annotation information of the fake video samples is used to characterize the editing traces of the fake video samples.

[0192] The parameter update module 5552 is used to update the parameters of the video detection model based on the real video samples, the fake video samples, and the corresponding annotation information, so as to obtain the trained video detection model.

[0193] In some embodiments, the parameter update module 5552 is further configured to, when the video detection model includes a pre-trained model, update the parameters of the pre-trained model based on the real video samples, the fake video samples, and their corresponding annotation information to obtain an initially trained video detection model; add a classification network to the initially trained video detection model to obtain a video detection model to be trained; and update the parameters of the classification network in the video detection model to be trained based on the real video samples and the fake video samples to obtain the trained video detection model.

[0194] In some embodiments, the parameter update module 5552 is further configured to add noise to each sample in the sample set to obtain a noisy sample, wherein the sample set includes the real video sample and the fake video sample; perform denoising on the noisy sample using the pre-trained model to obtain the editing traces of the sample; construct a first loss function based on the editing traces of the sample and the annotation information corresponding to the sample; and update the parameters of the pre-trained model based on the first loss function to obtain the initially trained pre-trained model.

[0195] In some embodiments, the parameter update module 5552 is further configured to encode the noisy sample based on the M cascaded encoding layers included in the pre-trained model to obtain a first encoded feature, wherein M is a positive integer greater than 0; perform attention processing on the first encoded feature to obtain a second encoded feature; and perform decoding processing on the second encoded feature based on the M cascaded decoding layers included in the pre-trained model to obtain the edit traces of the sample.

[0196] In some embodiments, the parameter update module 5552 is further configured to determine the binary classification annotation information corresponding to each sample in the sample set, wherein the sample set includes the real video samples and the fake video samples; determine the encoding features of the samples based on the pre-trained model in the video detection model to be trained; classify the encoding features of the samples based on the classification network in the video detection model to be trained to obtain the classification result of the samples; and update the parameters of the classification network of the video detection model to be trained based on the classification result and the binary classification annotation information to obtain the trained video detection model.

[0197] In some embodiments, the parameter update module 5552 is further configured to perform the following processing through the pre-trained model of the video detection model to be trained: encode the sample through M cascaded coding layers included in the pre-trained model to obtain the third coding feature of the sample; and perform attention processing on the third coding feature to obtain the coding feature of the sample.

[0198] In some embodiments, the parameter update module 5552 is further configured to perform the following processing through the classification network in the video detection model to be trained: performing dimensionality reduction processing on the encoded features of the sample to obtain the dimensionality reduction features of the sample; and performing mapping processing on the dimensionality reduction features of the sample to obtain the classification result of the sample.

[0199] In some embodiments, the sample construction module 5551 is further configured to perform target recognition processing on the real video sample to obtain at least one frame of target image; and to perform editing processing on the at least one frame of target image to obtain the fake video sample.

[0200] In some embodiments, the sample construction module 5551 is further configured to perform the following processing for each frame of the target image: extracting a first feature point of the target image, adjusting the feature value of the first feature point, and determining the feature point corresponding to the adjusted feature value as a second feature point; performing triangulation processing on the first feature point and the second feature point respectively to obtain a first grid and a second grid; performing transformation processing on the target image based on the transformation relationship between the first grid and the second grid to obtain an edited image; and replacing the target image in the real video sample with the edited image to obtain the fake video sample.

[0201] In some embodiments, the sample construction module 5551 is further configured to determine a plurality of first images included in the real video sample, and to determine a second image corresponding to each of the first images from the fake video sample; to determine difference information between each first image and the corresponding second image; and to determine the plurality of difference information as differences between the real video sample and the fake video sample.

[0202] In some embodiments, the sample construction module 5551 is further configured to perform the following processing for any position in the first image: determine a blank image, wherein the size of the blank image is the same as the size of the first image; perform the following processing for each position in the blank image: determine a first pixel at the position in the first image, and determine a second pixel at the position in the second image; when there is a difference between the first pixel and the second pixel, assign the pixel at the position in the blank image to a first set pixel; when there is no difference between the first pixel and the second pixel, assign the pixel at the position in the blank image to a second set pixel; and determine the blank image after assignment as the difference information.

[0203] This application provides a computer program product, which includes computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to execute the training method of the video detection model described in this application.

[0204] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the training method of the video detection model provided in this application. For example, ... Figures 3A to 3H The training method for the video detection model is shown.

[0205] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0206] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0207] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0208] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0209] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A training method for a video detection model, characterized in that, The method comprises: acquiring a real video sample carrying label information, wherein the real video sample comprises multiple frames of images; editing at least one frame of image in the real video sample to obtain a false video sample; determining a difference between the real video sample and the false video sample, and determining label information of the false video sample based on the difference, wherein the label information of the false video sample is used to represent an editing trace of the false video sample; updating parameters of the video detection model based on the real video sample, the false video sample, and the corresponding label information respectively to obtain a trained video detection model.

2. The method of claim 1, wherein the updating of the parameters of the video detection model based on the real video sample, the false video sample, and the corresponding label information respectively to obtain the trained video detection model comprises: when the video detection model comprises a pre-trained model, updating parameters of the pre-trained model based on the real video sample, the false video sample, and the corresponding label information respectively to obtain an initially trained video detection model; adding a classification network in the initially trained video detection model to obtain a video detection model to be trained; updating parameters of the classification network in the video detection model to be trained based on the real video sample and the false video sample to obtain the trained video detection model.

3. The method of claim 2, wherein, the updating of the parameters of the pre-trained model based on the real video sample, the false video sample, and the corresponding label information respectively to obtain the initially trained video detection model comprises: performing noise adding processing on each sample in a sample set to obtain a noise-added sample, wherein the sample set comprises the real video sample and the false video sample; performing noise removing processing on the noise-added sample by the pre-trained model to obtain an editing trace of the sample; constructing a first loss function based on the editing trace of the sample and the label information corresponding to the sample; updating parameters of the pre-trained model based on the first loss function to obtain an initially trained pre-trained model.

4. The method of claim 3, wherein, the noise removing processing on the noise-added sample by the pre-trained model to obtain the editing trace of the sample comprises: performing encoding processing on the noise-added sample based on M cascaded encoding layers included in the pre-trained model to obtain first encoding features, wherein M is a positive integer greater than 0; performing attention processing on the first encoding features to obtain second encoding features; performing decoding processing on the second encoding features based on M cascaded decoding layers included in the pre-trained model to obtain the editing trace of the sample.

5. The method of claim 2, wherein, the updating of the parameters of the classification network in the video detection model to be trained based on the real video sample and the false video sample comprises: determining binary classification label information corresponding to each sample in a sample set, wherein the sample set comprises the real video sample and the false video sample; determine the encoding feature of the sample based on a pre-training model in the video detection model to be trained; perform classification processing on the encoding feature of the sample based on a classification network in the video detection model to be trained, to obtain a classification result of the sample; update parameters of the classification network of the video detection model to be trained based on the classification result and the binary classification annotation information, to obtain a trained video detection model.

6. The method of claim 5, wherein, The method further includes: perform the following processing by the pre-training model of the video detection model to be trained: perform encoding processing on the sample by M cascaded encoding layers included in the pre-training model, to obtain third encoding features of the sample; perform attention processing on the third encoding features, to obtain the encoding features of the sample.

7. The method of claim 5, wherein, The method further includes: perform the following processing by the classification network in the video detection model to be trained: perform dimension reduction processing on the encoding features of the sample, to obtain dimension reduction features of the sample; perform mapping processing on the dimension reduction features of the sample, to obtain the classification result of the sample.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: perform target recognition processing on the real video sample, to obtain at least one target image; perform editing processing on the at least one target image, to obtain the fake video sample.

9. The method of claim 8, wherein, The method further includes: perform the following processing for each target image: extract first feature points of the target image, and adjust feature values of the first feature points, to determine second feature points corresponding to the adjusted feature values; perform triangulation processing on the first feature points and the second feature points respectively, to obtain first meshes and second meshes; perform transformation processing on the target image based on a transformation relationship between the first meshes and the second meshes, to obtain an edited image; replace the target image in the real video sample with the edited image, to obtain the fake video sample.

10. The method of claim 1, wherein, The method further includes: determine a plurality of first images included in the real video sample, and determine a second image corresponding to each first image from the fake video sample; determine difference information between each first image and the corresponding second image; determine a plurality of the difference information as the difference between the real video sample and the fake video sample.

11. The method of claim 10, wherein, The method further includes: determine a blank image, where the blank image has the same size as the first image; perform the following processing for each position in the blank image: determine a first pixel point at the position in the first image, and determine a second pixel point at the position in the second image; when the first pixel point and the second pixel point are different, assign a pixel point at the position in the blank image as a first set pixel point; when the first pixel point and the second pixel point are not different, assign a pixel point at the position in the blank image as a second set pixel point determine the blank image after the assignment as the difference information. 12.A device for training a video detection model, characterized in that, The device comprises: a sample construction module configured to obtain a real video sample carrying annotation information, wherein the real video sample comprises multiple images; the sample construction module is further configured to edit at least one image in the real video sample to obtain a false video sample; the sample construction module is further configured to determine a difference between the real video sample and the false video sample, and determine annotation information of the false video sample based on the difference, wherein the annotation information of the false video sample is used to represent an editing trace of the false video sample; a parameter updating module configured to update parameters of the video detection model based on the real video sample, the false video sample, and the respective annotation information, to obtain a trained video detection model.

13. An electronic device, comprising: The electronic device comprises: a memory configured to store computer executable instructions; a processor configured to execute the computer executable instructions or computer programs stored in the memory, to implement the training method of the video detection model according to any one of claims 1 to 11.

14. A computer-readable storage medium storing computer-executable instructions or a computer program, wherein the computer-executable instructions or the computer program comprise the steps of: The computer executable instructions or computer programs are executed by the processor to implement the training method of the video detection model according to any one of claims 1 to 11. ​ 15. A computer program product comprising computer-executable instructions or a computer program, characterized in that, The computer executable instructions or computer programs are executed by the processor to implement the training method of the video detection model according to any one of claims 1 to 11.

Images