Face image manipulation trace detection method, computer device and readable storage medium
By combining a fixed weight convolution network and a cornerstone network with attention module, using gradient operation submodule, the detection method significantly improves the accuracy and efficiency of face image operation trace detection, and solves the face image in the prior art Forgetting difficult to identify problems.
Patent Information
- Application Number
- CN202111094844.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-17
- Publication Date
- 2024-12-31
- Estimated Expiration
- 2041-09-17
AI Technical Summary
The prior art is difficult to detect the operation traces of face images efficiently and accurately, making it difficult to identify fake images and affecting social security.
Using a face image operation trace detection method, by analyzing the input face image, using a convolutional network with a fixed weight and a cornerstone network with attention module, combined with a gradient operation submodule, capture and fusion features To identify operation trace characteristics, improve detection accuracy.
It significantly improves the accuracy and efficiency of facial image operation trace detection, and can more accurately identify forged traces in facial images, preventing improper image forgery.
Smart Images
Figure CN113762205B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and specifically relates to a method for detecting facial image manipulation traces, a computer device, and a readable storage medium. Background Art
[0002] Using image processing technology to edit and beautify photographed images is becoming more and more popular among young people. However, using image processing technology to achieve improper purposes, image forgery, has brought serious social problems. For example, using image processing technology to generate facial images of celebrities or politicians with arbitrary identities or semantics, and using these images to mislead the public and cause public opinion. Therefore, it is urgent to develop some detection schemes to determine the authenticity of face images. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide a facial image operation trace detection method, a computer device and a readable storage medium to solve the problem of more efficiently and accurately capturing operation traces in facial images to identify whether the facial image is a forged facial image.
[0004] In a first aspect, an embodiment of the present application provides a method for detecting facial image manipulation traces, comprising:
[0005] Parsing an input face image to obtain an initial feature tensor corresponding to the face image;
[0006] Inputting the initial feature tensor into a convolutional network with a layer of fixed weights to perform a convolution operation to obtain a refined feature tensor;
[0007] The refined feature tensor is input into the cornerstone network to obtain the operation trace feature of the face image.
[0008] Among them, the convolutional network with a layer of fixed weights is a gradient operator.
[0009] In a second aspect, an embodiment of the present application provides a method for detecting facial image manipulation traces, including:
[0010] Parsing an input face image to obtain an initial feature tensor corresponding to the face image;
[0011] Inputting the initial feature tensor into a cornerstone network with an attention module to obtain operation trace features of the face image;
[0012] The cornerstone network with the attention module includes a plurality of cornerstone nodes, each of which is embedded with an attention module, and the attention module is used to refine the feature tensor input into each cornerstone node to finally obtain the operation trace features in the face image;
[0013] Among them, the attention module includes a maximum pooling operation module, an average pooling operation module and a gradient sub-module. The feature tensor output by the attention module includes the feature tensor output by the maximum pooling module, the average pooling operation module and the gradient sub-module. The operation sub-module is a processing module for performing a convolution operation on a convolutional network with a layer of fixed weights.
[0014] In a third aspect, an embodiment of the present application provides a method for detecting facial image manipulation traces, comprising:
[0015] Parsing an input face image to obtain an initial feature tensor corresponding to the face image;
[0016] Inputting the initial feature tensor into a convolutional network with a layer of fixed weights to perform a convolution operation to obtain a refined feature tensor, and inputting the refined feature tensor into a cornerstone network to obtain a first operation trace feature of the face image;
[0017] Wherein, the convolutional network with a layer of fixed weights is a gradient operator;
[0018] Inputting the initial feature tensor into a cornerstone network with an attention module to obtain a second operation trace feature of the face image;
[0019] The cornerstone network includes a plurality of cornerstone nodes, each of which is embedded with an attention module, and the attention module is used to refine the feature tensor input into each cornerstone node;
[0020] The attention module includes a maximum pooling operation module, an average pooling operation module and a gradient submodule, and the feature tensor output by the attention module includes the feature tensor output by the maximum pooling module, the average pooling operation module and the gradient submodule;
[0021] The first operation trace feature and the second operation trace feature are fused to obtain a final output.
[0022] Furthermore, the cornerstone network is a ResNet-18 network.
[0023] Furthermore, the maximum pooling operation module and the average pooling operation module are used to capture the manipulation trace association between the primary color channels of the face image, and generate a channel attention map through a shared network to obtain a first sub-operation trace feature;
[0024] The gradient submodule is used to further capture the operation trace feature of the face image to obtain a second sub-operation trace feature;
[0025] The first sub-operation trace feature and the second sub-operation trace feature are merged to obtain the operation trace feature of the face image output by the basic network with attention module.
[0026] Furthermore, the method for detecting facial image manipulation traces also includes:
[0027] Performing weighted processing on the second sub-operation trace feature obtained by the gradient sub-module by using a preset weight to obtain a weighted second sub-operation trace feature;
[0028] The first sub-operation trace feature and the second sub-operation trace feature are merged as follows:
[0029] The first sub-operation trace feature and the second sub-operation trace feature are fused through a sigmoid function.
[0030] Furthermore, the shared network includes a 1×1 convolutional layer and a ReLU activation function.
[0031] Furthermore, the first operation trace feature and the second operation trace feature are fused to obtain a final output as follows:
[0032] The first operation trace feature and the second operation trace feature are fused by adding each pixel one by one.
[0033] In a fourth aspect, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory is used to store one or more programs;
[0034] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-described methods.
[0035] In a fifth aspect, an embodiment of the present application provides a computer device and a readable storage medium, including a computer program, which, when executed on a computer, enables the computer to execute the method described above.
[0036] In an embodiment of the present application, a preprocessing module is added before the cornerstone network. The preprocessing module embeds the gradient operator into the convolution layer, so that it acts as a convolution layer with a fixed kernel coefficient. It can refine the feature tensor of the face image and deeply capture the manipulation trace features left by the face forgery technology. Then, the refined feature tensor is input into the cornerstone network, which can greatly improve the detection performance of the deep network for face manipulation traces.
[0037] In another embodiment of the present application, by embedding the gradient operator into the cornerstone network, the facial image features can be further refined to obtain the operation trace features of the facial image. Compared with the single cornerstone network detection, the cornerstone network embedded with the gradient operator in the embodiment of the present application can more accurately identify the facial manipulation traces.
[0038] In another embodiment of the present application, a deep network model with two channels is included, wherein one of the deep network models includes a preprocessing module and a cornerstone network, wherein the preprocessing module embeds a gradient operator into a convolutional layer so that it serves as a convolutional layer with a fixed kernel coefficient, thereby refining the feature tensor of the face image and deeply capturing the manipulation trace features left by the face forgery technology; the other deep network model is a cornerstone network with an attention module, specifically, the gradient operator is embedded into each cornerstone node of the cornerstone network to supplement the manipulation trace features left by the face forgery technology, and the features output by the two deep network models are fused to obtain accurate identification of the manipulation traces in the face image. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0040] Figure 1 This is a schematic diagram of the structure of the cornerstone network provided in the embodiment of the present application.
[0041] Figure 2 This is a flowchart of the method for detecting facial image manipulation traces provided by the first embodiment of the present application.
[0042] Figure 3 This is a schematic diagram of the face image manipulation trace detection model provided in the first embodiment of the present application.
[0043] Figure 4 This is a flowchart of a method for detecting facial image manipulation traces provided by the second embodiment of the present application.
[0044] Figure 5 This is a schematic diagram of a face image manipulation trace detection model provided in the second embodiment of the present application.
[0045] Figure 6 This is a flowchart of a method for detecting facial image manipulation traces provided in the third embodiment of the present application.
[0046] Figure 7 This is a schematic diagram of a facial image manipulation trace detection model provided in the third embodiment of the present application.
[0047] Figure 8 This is a schematic diagram of the structure of a facial image manipulation close detection device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0049] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0050] The following describes in detail the method for generating an interpolation frame provided in the embodiment of the present application through specific embodiments and application scenarios in conjunction with the accompanying drawings.
[0051] Cornerstone Networks, such as Figure 1 As shown, it includes a 7×7 convolutional layer, a batch normalization ReLU activation function, a 3×3 maximum pool, and several cornerstone nodes. In one embodiment, the cornerstone network uses a ResNet-18 network.
[0052] The sigmoid function, also called the Logistic function, is used for the output of hidden layer neurons. Its value range is (0,1). It can map a real number to the interval (0,1).
[0053] like Figure 2 and Figure 3 As shown, an embodiment of the present invention provides a method for detecting facial image manipulation traces, comprising:
[0054] Step 201: parse the input face image to obtain an initial feature tensor corresponding to the face image.
[0055] Step 202: Input the initial feature tensor into a convolutional network with a layer of fixed weights for convolution operation to obtain a refined feature tensor, wherein the convolutional network with a layer of fixed weights is a gradient operator.
[0056] The face image is parsed to obtain an initial feature tensor corresponding to the face image, and the initial feature tensor is convolved with a gradient operator of a convolutional network with a fixed weight layer to obtain a refined feature tensor.
[0057] Step 203: Input the refined feature tensor into the cornerstone network to obtain the operation trace features of the face image.
[0058] The refined feature tensor is input into the cornerstone network, that is, the refined feature tensor is further analyzed by the cornerstone network. If the face image is a fake face image that has been artificially manipulated, the manipulation trace features of the face image will be obtained.
[0059] In another embodiment, Figure 4 As shown, a method for detecting operation traces of a face image is provided, comprising:
[0060] Step 301: parse the input face image to obtain an initial feature tensor corresponding to the face image.
[0061] Step 302: Input the initial feature tensor into a cornerstone network with an attention module to obtain the operation trace feature of the face image. The cornerstone network with an attention module includes a plurality of cornerstone nodes, each of which is embedded in an attention module, and the attention module is used to refine the feature tensor input into each cornerstone node to finally obtain the operation trace feature in the face image. The attention module includes a maximum pooling operation module, an average pooling operation module, and a gradient submodule, and the feature tensor output by the attention module includes the feature tensors output by the maximum pooling module, the average pooling operation module, and the gradient submodule.
[0062] like Figure 5 As shown, the facial image manipulation trace recognition algorithm of this embodiment is an improvement on the cornerstone network, specifically, the attention module is embedded into each cornerstone node in the cornerstone network. The embedding of the attention module enables each cornerstone node in the cornerstone network to have better feature recognition capability. Since the attention module includes a maximum pooling operation module, an average pooling operation module and a gradient submodule. The facial image manipulation trace recognition algorithm of this embodiment is essentially to embed the gradient sub-algorithm into the cornerstone network, and the new deep network formed has better feature recognition capability.
[0063] Further, at each cornerstone node, the maximum pooling operation module and the average pooling operation module in the attention module are used to capture the manipulation trace association between the primary color channels of the face image, and generate a channel attention map through a shared network to obtain the first sub-operation trace feature. The shared network here includes a 1×1 convolution layer and a ReLU activation function.
[0064] The gradient submodule is used to further capture the operation trace feature of the face image to obtain a second sub-operation trace feature;
[0065] The first sub-operation trace feature is fused with the second sub-operation trace feature to obtain the refined facial feature tensor output by each cornerstone node. The output of the previous cornerstone node is used as the input of the next cornerstone node. The feature output by the last cornerstone node is average pooled and obtained as the final output of the model, which can reflect the manipulation traces of the facial image.
[0066] Furthermore, the first sub-operation trace feature and the second sub-operation trace feature are merged in the following manner:
[0067] Performing weighted processing on the second sub-operation trace feature obtained by the gradient sub-module by using a preset weight to obtain a weighted second sub-operation trace feature;
[0068] The first sub-operation trace feature and the second sub-operation trace feature are merged as follows:
[0069] The first sub-operation trace feature and the second sub-operation trace feature are fused through a sigmoid function.
[0070] like Figure 6 and Figure 7 As shown, in yet another embodiment, a method for detecting traces of facial image manipulation is provided, comprising:
[0071] Step 401: parse the input face image to obtain an initial feature tensor corresponding to the face image.
[0072] Step 402: Input the initial feature tensor into a convolutional network with a layer of fixed weights for convolution operation to obtain a refined feature tensor, and input the refined feature tensor into a cornerstone network to obtain a first operation trace feature of the face image, wherein the convolutional network with a layer of fixed weights is a gradient operation.
[0073] Step 403: Input the initial feature tensor into a cornerstone network with an attention module to obtain a second operation trace feature of the face image, wherein the cornerstone network includes multiple cornerstone nodes, each of the cornerstone nodes is embedded in an attention module, and the attention module is used to refine the feature tensor input into each cornerstone node; wherein the attention module includes a maximum pooling operation module, an average pooling operation module and a gradient submodule, and the feature tensor output by the attention module includes the feature tensors output by the maximum pooling module, the average pooling operation module and the gradient submodule.
[0074] Step 404: Fusing the first operation trace feature and the second operation trace feature to obtain a final output.
[0075] In this embodiment, step 402 is the same as the feature recognition method in the first embodiment, and step 403 is the same as the recognition method in the second embodiment. Therefore, these two steps are not repeated in this embodiment.
[0076] The method for identifying manipulation traces of facial images of the present embodiment identifies the same facial image through a two-channel deep network, and then fuses the facial image features obtained from the two channels to more accurately identify the manipulation trace features corresponding to the facial image.
[0077] Specifically, step 404 includes: fusing the first operation trace feature and the second operation trace feature by adding them pixel by pixel.
[0078] In one embodiment, a facial image manipulation trace detection device is provided, comprising:
[0079] The face image input module is used to parse the input face image and obtain the initial feature tensor corresponding to the face image.
[0080] The gradient subconvolution preprocessing module is used to input the initial feature tensor into a convolution network with a layer of fixed weights to perform a convolution operation to obtain a refined feature tensor.
[0081] A cornerstone network module is used to input the refined feature tensor into the cornerstone network to obtain the operation trace features of the face image.
[0082] Among them, the convolutional network with a layer of fixed weights is a gradient operator.
[0083] In another embodiment, a facial image manipulation trace detection device is provided, comprising:
[0084] A face image input module, used to analyze the input face image and obtain an initial feature tensor corresponding to the face image;
[0085] A cornerstone module with an attention module is used to input the initial feature tensor into a cornerstone network with an attention module to obtain the operation trace features of the face image; wherein the cornerstone network with an attention module includes multiple cornerstone nodes, each of which is embedded in an attention module, and the attention module is used to refine the feature tensor input into each cornerstone node to ultimately obtain the operation trace features in the face image; wherein the attention module includes a maximum pooling operation module, an average pooling operation module and a gradient submodule, and the feature tensor output by the attention module includes the feature tensors output by the maximum pooling module, the average pooling operation module and the gradient submodule.
[0086] In another embodiment, Figure 8 As shown, a facial image manipulation trace detection device is provided, comprising:
[0087] The face image input module 802 is used to analyze the input face image and obtain the initial feature tensor corresponding to the face image.
[0088] The gradient sub-preprocessing module 804 is used to input the initial feature tensor into a convolutional network with a layer of fixed weights to perform a convolution operation to obtain a refined feature tensor, and input the refined feature tensor into a cornerstone network to obtain the first operation trace feature of the face image; wherein the convolutional network with a layer of fixed weights is a gradient operator.
[0089] The cornerstone module 806 with an attention module is used to input the initial feature tensor into the cornerstone network with an attention module to obtain the second operation trace feature of the face image; wherein the cornerstone network includes multiple cornerstone nodes, each of the cornerstone nodes is embedded in an attention module, and the attention module is used to refine the feature tensor input into each cornerstone node; wherein the attention module includes a maximum pooling operation module, an average pooling operation module and a gradient submodule, and the feature tensor output by the attention module includes the feature tensors output by the maximum pooling module, the average pooling operation module and the gradient submodule.
[0090] The feature fusion module 808 fuses the first operation trace feature and the second operation trace feature to obtain a final output.
[0091] Optionally, an embodiment of the present application also provides an electronic device, including one or more processors and a memory for storing one or more programs. When the one or more programs are executed by the processor, the various processes of the above-mentioned facial image manipulation trace detection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be repeated here.
[0092] An embodiment of the present application also provides a readable storage medium, which includes a computer program. When the computer program is executed on a computer, it implements the various processes of the above-mentioned facial image operation trace detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0093] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM) or a random access memory (RAM).
[0094] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0095] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A method for detecting operation traces of facial images, characterized in that: include: Parsing an input face image to obtain an initial feature tensor corresponding to the face image; Inputting the initial feature tensor into a cornerstone network with an attention module to obtain operation trace features of the face image; The cornerstone network with the attention module includes a plurality of cornerstone nodes, each of which is embedded with an attention module, and the attention module is used to refine the feature tensor input into each cornerstone node to finally obtain the operation trace features in the face image; Among them, the attention module includes a maximum pooling operation module, an average pooling operation module and a gradient operation sub-module. The feature tensor output by the attention module includes the feature tensor output by the maximum pooling module, the average pooling operation module and the gradient operation sub-module. The gradient operation sub-module is a processing module for performing convolution operations on a convolutional network with a layer of fixed weights.
2. A method for detecting facial image manipulation traces, characterized in that: include: Parsing an input face image to obtain an initial feature tensor corresponding to the face image; Inputting the initial feature tensor into a convolutional network with a layer of fixed weights to perform a convolution operation to obtain a refined feature tensor, and inputting the refined feature tensor into a cornerstone network to obtain a first operation trace feature of the face image; Wherein, the convolutional network with a layer of fixed weights is a gradient operator; Inputting the initial feature tensor into a cornerstone network with an attention module to obtain a second operation trace feature of the face image; The cornerstone network includes a plurality of cornerstone nodes, each of which is embedded with an attention module, and the attention module is used to refine the feature tensor input into each cornerstone node; The attention module includes a maximum pooling operation module, an average pooling operation module and a gradient operation submodule, and the feature tensor output by the attention module includes the feature tensor output by the maximum pooling module, the average pooling operation module and the gradient operation submodule; The first operation trace feature and the second operation trace feature are fused to obtain a final output.
3. The method for detecting facial image manipulation traces according to any one of claims 1 to 2, characterized in that The cornerstone network is the ResNet-18 network.
4. The method for detecting facial image manipulation traces according to claim 1 or 2, characterized in that: The maximum pooling operation module and the average pooling operation module are used to capture the manipulation trace association between the primary color channels of the face image, and generate a channel attention map through a shared network to obtain a first sub-operation trace feature; The gradient operation submodule is used to further capture the operation trace feature of the face image to obtain a second sub-operation trace feature; The first sub-operation trace feature and the second sub-operation trace feature are merged to obtain the operation trace feature of the face image output by the basic network with attention module.
5. The method for detecting facial image manipulation traces according to claim 4, characterized in that: The method further comprises: Performing weighted processing on the second sub-operation trace feature obtained by the gradient operation submodule by using a preset weight to obtain a weighted second sub-operation trace feature; The first sub-operation trace feature and the second sub-operation trace feature are merged as follows: The first sub-operation trace feature and the second sub-operation trace feature are fused through a sigmoid function.
6. The method for detecting facial image manipulation traces according to claim 4, characterized in that: The shared network includes a 1×1 convolutional layer and a ReLU activation function.
7. The method for detecting facial image manipulation traces according to claim 2, characterized in that: The first operation trace feature and the second operation trace feature are fused to obtain the final output: The first operation trace feature and the second operation trace feature are fused by adding each pixel one by one.
8. A computer device comprising a memory and a processor, wherein: The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
9. A readable storage medium, characterized in that: The invention comprises a computer program, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 7.