Method, device, electronic device and storage medium for identifying synthetic face images

Abstract

The present application provides a method, device, electronic device and storage medium for identifying synthetic facial images. The identification method includes: performing image processing on a target facial image to determine a pixel cluster and a feature cluster of the target facial image; performing category attention calculation on each pixel set in the pixel cluster to determine a synthetic category attention feature matrix of each pixel set; performing position attention calculation on each feature set in the feature cluster to determine a synthetic position attention feature matrix of each feature set; performing aggregation processing on multiple synthetic category attention feature matrices and multiple synthetic position attention feature matrices to obtain a total attention feature vector of the target facial image; performing full connection processing on the total attention feature vector to determine the synthesis method category and synthesis position of the target facial image. Thus, the efficiency and accuracy of identifying synthetic facial images are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for identifying synthetic facial images. Background Art

[0002] The widespread application of face recognition technology in people's lives has brought great convenience to people's lives. However, many face synthesis technologies have recently emerged, which has greatly threatened face recognition. Synthesized faces can easily deceive face recognition systems and bring serious hidden dangers to people's property. There are roughly two types of methods for synthesizing faces. One category is to use the stylegan algorithm for synthesis, and the other is to generate using the diffusion model diffusion method. Most methods for identifying these synthetic faces use decoder-based technology to find flaws in synthetic faces, such as blurred edges, blocky spots, background errors, hair texture, etc., but these methods will become invalid as the synthesis technology improves. Therefore, how to quickly identify whether the face accepted by the system is a real face or a computer-synthesized face has become a technical issue that cannot be underestimated. Summary of the invention

[0003] In view of this, the purpose of the present application is to provide a method, device, electronic device and storage medium for identifying synthetic facial images, by utilizing pixel clusters and feature clusters to determine the synthetic category attention feature matrix and the synthetic position attention feature matrix, and aggregating the synthetic category attention feature matrix and the synthetic position attention feature matrix so that different matrices can exchange information, thereby determining the synthesis method category and synthesis position of the facial image, thereby improving the efficiency and accuracy of identifying synthetic facial images.

[0004] The present application provides a method for identifying a synthetic face image, the method comprising:

[0005] Performing image processing on the target face image to determine pixel clusters and feature clusters of the target face image;

[0006] Performing category attention calculation on each pixel set in the pixel cluster to determine a synthetic category attention feature matrix of each pixel set, and performing position attention calculation on each feature set in the feature cluster to determine a synthetic position attention feature matrix of each feature set;

[0007] The multiple synthesis category attention feature matrices and the multiple synthesis position attention feature matrices are aggregated to obtain the total attention feature vector of the target face image, and the total attention feature vector is fully connected to determine the synthesis method category and synthesis position of the target face image.

[0008] In a possible implementation manner, performing image processing on the target facial image to determine the pixel clusters and feature clusters of the target facial image includes:

[0009] Determining the pixel cluster in the target face image according to a pixel set under a target window centered on each pixel point;

[0010] Convolution processing, pooling processing and normalization processing are performed on each pixel set in the pixel cluster to determine a feature set corresponding to each pixel set, and the feature cluster is determined based on multiple feature sets.

[0011] In a possible implementation manner, performing category attention calculation on each pixel set in the pixel cluster to determine a synthetic category attention feature matrix of each pixel set in the pixel cluster includes:

[0012] For each pixel set in the pixel cluster, based on the pixel set, the hyperparameter matrix and the pixel cluster, a query embedding vector corresponding to the pixel set is determined, and the pixel set, the hyperparameter matrix and the query embedding vector corresponding to the pixel set are processed based on the activation function to generate a synthetic category attention feature matrix for the pixel set.

[0013] In a possible implementation manner, performing position attention calculation on each feature set in the feature cluster to determine a position attention feature matrix of each feature set includes:

[0014] For each feature set in the feature cluster, the hyperparameter position matrix, the feature set and the parameter limit values ​​of the target window are processed based on the activation function to generate a synthetic position attention feature matrix of the feature set.

[0015] In a possible implementation manner, the performing full connection processing on the total attention feature vector to determine the synthesis method category and synthesis position of the target face image includes:

[0016] Splitting the total attention feature vector to determine a target synthesis category attention feature vector and a target synthesis position attention feature vector;

[0017] Performing result mapping processing on the target synthesis category attention feature vector based on a fully connected mapping function to map out the category of the synthesis method of the target face image;

[0018] The target synthesis position attention feature vector is mapped based on the fully connected mapping function to determine the synthesis position of the target face image.

[0019] In a possible implementation manner, the total attention feature vector of the target face image is determined by the following formula:

[0020]

[0021] Among them, S i is the synthetic category attention feature matrix, P i is the synthetic position attention feature matrix, d is the scaling factor, W is the learnable parameter matrix, m is the number of pixel sets, and A is the total attention feature vector.

[0022] The embodiment of the present application also provides a synthetic face image identification device, the identification device comprising:

[0023] An image processing module, used to perform image processing on a target face image to determine a pixel cluster and a feature cluster of the target face image;

[0024] An attention calculation module, used to perform category attention calculation on each pixel set in the pixel cluster to determine a synthetic category attention feature matrix of each pixel set, and to perform position attention calculation on each feature set in the feature cluster to determine a synthetic position attention feature matrix of each feature set;

[0025] The synthesis determination module is used to aggregate the multiple synthesis category attention feature matrices and the multiple synthesis position attention feature matrices to obtain the total attention feature vector of the target face image, perform full connection processing on the total attention feature vector, and determine the synthesis method category and synthesis position of the target face image.

[0026] In a possible implementation manner, when the image processing module is used to perform image processing on the target face image to determine the pixel clusters and feature clusters of the target face image, the image processing module is specifically used to:

[0027] Determining the pixel cluster in the target face image according to a pixel set under a target window centered on each pixel point;

[0028] Convolution processing, pooling processing and normalization processing are performed on each pixel set in the pixel cluster to determine a feature set corresponding to each pixel set, and the feature cluster is determined based on multiple feature sets.

[0029] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for identifying a synthetic facial image as described above are performed.

[0030] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the synthetic facial image identification method as described above are executed.

[0031] The identification method, device, electronic device and storage medium of synthetic face images provided in the embodiment of the present application, the identification method includes: performing image processing on the target face image to determine the pixel cluster and feature cluster of the target face image; performing category attention calculation on each pixel set in the pixel cluster to determine the synthetic category attention feature matrix of each pixel set, performing position attention calculation on each feature set in the feature cluster to determine the synthetic position attention feature matrix of each feature set; aggregating multiple synthetic category attention feature matrices and multiple synthetic position attention feature matrices to obtain the total attention feature vector of the target face image, performing full connection processing on the total attention feature vector to determine the synthesis method category and synthesis position of the target face image. By using pixel clusters and feature clusters to determine the synthetic category attention feature matrix and the synthetic position attention feature matrix, and aggregating the synthetic category attention feature matrix and the synthetic position attention feature matrix so that different matrices can exchange information, the synthesis method category and synthesis position of the face image are determined, thereby improving the efficiency and accuracy of identification of synthetic face images.

[0032] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 One of the flow charts of a synthetic face image identification method provided in an embodiment of the present application;

[0035] Figure 2 A second flowchart of a synthetic face image identification method provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the structure of a synthetic face image identification device provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0039] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0040] In order to enable those skilled in the art to use the contents of this application, the following implementation is given in combination with the specific application scenario of "identifying facial images". For those skilled in the art, the general principles defined here can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application.

[0041] The following methods, devices, electronic devices or computer-readable storage media of the embodiments of the present application can be applied to any scenario where facial images need to be identified. The embodiments of the present application are not limited to specific application scenarios. Any scheme of identifying synthetic facial images using the methods, devices, electronic devices and storage media provided by the embodiments of the present application is within the scope of protection of the present application.

[0042] First, the application scenarios to which the present application is applicable are introduced. The present application can be applied in the field of image processing technology.

[0043] Research has found that the widespread application of face recognition technology in people's lives has brought great convenience to people's lives. However, many face synthesis technologies have recently emerged, which has greatly threatened face recognition. Synthesized faces can easily deceive face recognition systems and bring serious hidden dangers to people's property. There are roughly two types of methods for synthesizing faces. One category is to use the stylegan algorithm for synthesis, and the other is to generate using the diffusion model diffusion method. Most methods for identifying these synthetic faces use decoder-based technology to find flaws in synthetic faces, such as blurred edges, blocky spots, background errors, hair texture, etc., but these methods will become invalid as the synthesis technology improves. Therefore, how to quickly identify whether the face accepted by the system is a real face or a computer-synthesized face has become a technical issue that cannot be underestimated.

[0044] Based on this, an embodiment of the present application provides a method for identifying synthetic facial images, by utilizing pixel clusters and feature clusters to determine a synthetic category attention feature matrix and a synthetic position attention feature matrix, and aggregating the synthetic category attention feature matrix and the synthetic position attention feature matrix so that different matrices can exchange information, thereby determining the synthesis method category and synthesis position of the facial image, thereby improving the efficiency and accuracy of identifying synthetic facial images.

[0045] See also Figure 1 , Figure 1 This is one of the flow charts of a synthetic face image identification method provided in an embodiment of the present application. Figure 1 As shown in , the identification method provided by the embodiment of the present application includes:

[0046] S101: performing image processing on a target facial image to determine pixel clusters and feature clusters of the target facial image.

[0047] In this step, a target face image is acquired, and image processing is performed on the target face image to determine pixel clusters and feature clusters of the target face image.

[0048] The pixel set is composed of multiple pixel point sets of the target face image, and the feature cluster is obtained by performing feature processing on the pixel point set.

[0049] The target face image may be a color synthesized face image or a non-synthesized face image, and this part is not limited.

[0050] In a possible implementation manner, performing image processing on the target facial image to determine the pixel clusters and feature clusters of the target facial image includes:

[0051] A: Determine the pixel cluster in the target face image according to a pixel set under a target window centered on each pixel point.

[0052] Here, in the target face image, a pixel cluster is determined according to a pixel set under a target window centered on each pixel point. Specifically, the input color synthetic face image I, Where C = 3 represents a 3-channel color image, H and W represent the height and width of the image respectively. Define the pixel set of a specific size window centered on the pixel point (i, j) as x ij , the pixel set centered at each pixel forms a pixel cluster X.

[0053] B: Perform convolution processing, pooling processing and normalization processing on each pixel set in the pixel cluster to determine a feature set corresponding to each pixel set, and determine the feature cluster based on multiple feature sets.

[0054] Here, convolution processing, pooling processing and normalization processing are performed on each pixel set in the pixel cluster to determine the feature set corresponding to each pixel set, and multiple feature sets constitute a feature cluster.

[0055] Among them, the pixel set is processed by the following formula to determine the feature set:

[0056] y ij =Normlize(Pooling(G-Conv(x ij )))

[0057] Among them, y ij is the feature set, x ij is a collection of pixels, G-Conv stands for gated convolution, Pooling stands for pooling, and Normalize stands for normalization operation.

[0058] Among them, the specific implementation of gated convolution is:

[0059] G-Conv = λ(Conv(W a x ij ))+W b x ij

[0060]

[0061] Among them, λ is the gating parameter, which determines whether the convolution is involved in the operation, W a ,Wb ,W c are the mapping parameter matrices, and σ is the activation function, such as the relu function.

[0062] S102: Perform category attention calculation on each pixel set in the pixel cluster to determine a synthetic category attention feature matrix for each pixel set, and perform position attention calculation on each feature set in the feature cluster to determine a synthetic position attention feature matrix for each feature set.

[0063] In this step, category attention calculation is performed on each pixel set in the pixel cluster to determine the synthetic category attention feature matrix of each pixel set, and position attention calculation is performed on each feature set in the feature cluster to determine the synthetic position attention feature matrix of each feature set.

[0064] The main purpose of calculating the category attention on the pixel clusters is to distinguish whether the face is synthesized by the stylegan algorithm or the diffusion model. The main purpose of calculating the position attention on the feature image is to find the locations where there are suspicious signs of synthesis in the image background, the surrounding of the face, and the facial organs.

[0065] Here, if neither the synthetic category attention feature matrix nor the synthetic position attention feature matrix is ​​determined, it means that the target face image is not a synthetic image.

[0066] In this scheme, the pixel cluster X and the feature cluster Y are respectively sent to the subsequent fine-grained attention module (category attention) and the coarse-grained attention module (position attention) to extract forged face features at different granularities.

[0067] In a possible implementation manner, performing category attention calculation on each pixel set in the pixel cluster to determine a synthetic category attention feature matrix of each pixel set in the pixel cluster includes:

[0068] For each pixel set in the pixel cluster, based on the pixel set, the hyperparameter matrix and the pixel cluster, a query embedding vector corresponding to the pixel set is determined, and the pixel set, the hyperparameter matrix and the query embedding vector corresponding to the pixel set are processed based on the activation function to generate a synthetic category attention feature matrix for the pixel set.

[0069] Here, for each pixel set in the pixel cluster, the query embedding vector corresponding to the pixel set is determined based on the pixel set, the hyperparameter matrix and the pixel cluster, and the pixel set, the hyperparameter matrix and the query embedding vector corresponding to the pixel set are processed according to the activation function to generate a synthetic category attention feature matrix for the pixel set.

[0070] Among them, due to x ij It is pixel-level data, representing the lowest level of image data information, so a fine-grained category attention module is required to extract forged features.

[0071] The entire forged image can be expressed as: X = {X 1 , X 2 , …, X m},in:

[0072] X 1 ={x 11 , x 12 , …, x 1n}

[0073] X 2 ={x 21 , x 22 , …, x 2n}

[0074] …

[0075] X m ={x m1 , x m2 , …, x mn}

[0076] m=H,n=W

[0077] Here, for each X i Perform attention calculation to obtain the category attention feature matrix S i :

[0078]

[0079] Among them, W Q , W K , W V is a learnable hyperparameter matrix that maps different feature maps to feature vectors of the same dimension and different functions for calculation. Q X i The effect of irrelevant information aggregation is added with the query embedding vector Q i As a supplement to the forged information query capability, the additional overhead is negligible.

[0080] Q i =Xi W Q X

[0081] In a possible implementation manner, performing position attention calculation on each feature set in the feature cluster to determine a position attention feature matrix of each feature set includes:

[0082] For each feature set in the feature cluster, the hyperparameter position matrix, the feature set and the parameter limit values ​​of the target window are processed based on the activation function to generate a synthetic position attention feature matrix of the feature set.

[0083] Here, for each feature set in the feature cluster, the hyperparameter position matrix, the feature set, and the parameter limit values ​​of the target window are processed according to the activation function to generate a synthetic position attention feature matrix of the feature set.

[0084] Among them, due to the pooling feature set y ij After the gated convolution operation, it is already a feature-level vector, so only coarse-grained position attention calculation is needed to obtain the coordinates of the forged position.

[0085] Here, the feature set y ij , then the feature cluster of the whole picture is Y={Y 1 , Y 2 , …, Y n},in:

[0086] Y 1 ={y 11 ,y 12 , …, y 1n}

[0087] Y 2 ={y 21 ,y 22 , …, y 2n}

[0088] …

[0089] Y m ={y m1 ,y m2 , …, y mn}

[0090] m=H,n=W

[0091] The feature map set y is given by the following formula ij Perform position attention calculation to obtain the position attention feature matrix P i :

[0092] P i=softMax(κlogT×W P Y i )Y i

[0093] Where W P is a learnable hyperparameter position matrix, which represents the position of the synthetic points that may appear in the feature map position. The feature map can be mapped to a feature vector with position information of the same dimension for calculation. In order to prevent the position boundary from crossing the boundary, logT is added to limit the position boundary, T = ||L(m,n)||-||L(δ(i,j))||, ||T|| represents the area of ​​the target window, δ represents the padding boundary operation, and κ is a scaling factor, which is a learnable parameter.

[0094] S103: Aggregate the multiple synthesis category attention feature matrices and the multiple synthesis position attention feature matrices to obtain the total attention feature vector of the target face image, perform full connection processing on the total attention feature vector, and determine the synthesis method category and synthesis position of the target face image.

[0095] In this step, multiple synthesis category attention feature matrices and multiple synthesis position attention feature matrices are aggregated to obtain a total attention feature vector of the target face image, and the total attention feature vector is fully connected to determine the synthesis method category and synthesis position of the target face image.

[0096] In a possible implementation manner, the total attention feature vector of the target face image is determined by the following formula:

[0097]

[0098] Among them, S i is the synthetic category attention feature matrix, P i is the synthetic position attention feature matrix, d is the scaling factor, W is the learnable parameter matrix, m is the number of pixel sets, and A is the total attention feature vector.

[0099] Here, different features are grouped together, and through the attention mechanism, the information features in different matrices are interacted, so that useful information in categories and positions can flow.

[0100] For further information, see Figure 2 , Figure 2 This is a second flow chart of a synthetic face image identification method provided in an embodiment of the present application. Figure 2 As shown, the total attention feature vector is fully connected to determine the synthesis method category and synthesis position of the target face image, including:

[0101] S201: Split the total attention feature vector to determine a target synthesis category attention feature vector and a target synthesis position attention feature vector.

[0102] In this step, the total attention feature vector is split according to the processing task to determine the target synthesis category attention feature vector and the target synthesis position attention feature vector.

[0103] Among them, the process of splitting the total attention feature vector is as follows:

[0104] [A X |A Y ]=Split(A)

[0105] Among them, A is the total attention feature vector, Split is the split operation, and in this solution, it is to process the task equally, A X is the target synthetic category attention feature vector, A Y Synthesize position attention feature vector for the target.

[0106] S202: Perform result mapping processing on the target synthesis category attention feature vector based on the fully connected mapping function to map out the category of the synthesis method of the target face image.

[0107] In this step, the target synthesis category attention feature vector is mapped according to the fully connected mapping function to map out the category of the synthesis method of the target face image.

[0108] Among them, the categories of synthesis methods are synthesizing corresponding categories using the stylegan algorithm and synthesizing corresponding categories using the diffusionmodel algorithm.

[0109] S203: Perform result mapping processing on the target synthesis position attention feature vector based on the fully connected mapping function to determine the synthesis position of the target face image.

[0110] In this step, the target synthesis position attention feature vector is mapped according to the fully connected mapping function to determine the synthesis position of the target face image.

[0111] Among them, the target synthesis category attention feature vector A X And the target synthesis position attention feature vector A Y The results are mapped respectively to obtain the final synthesis category C and synthesis position R.

[0112] Among them, the type of synthesis position and method is determined by the following formula, FCN X ,FCN YFor the fully connected mapping function:

[0113] C=FCN X (A X )

[0114] R=FCN Y (A Y )

[0115] The embodiment of the present application provides a method for identifying a synthetic face image, the identification method comprising: performing image processing on a target face image to determine a pixel cluster and a feature cluster of the target face image; performing category attention calculation on each pixel set in the pixel cluster to determine a synthetic category attention feature matrix of each pixel set; performing position attention calculation on each feature set in the feature cluster to determine a synthetic position attention feature matrix of each feature set; aggregating multiple synthetic category attention feature matrices and multiple synthetic position attention feature matrices to obtain a total attention feature vector of the target face image, performing full connection processing on the total attention feature vector to determine the synthesis method category and synthesis position of the target face image. By using pixel clusters and feature clusters to determine the synthetic category attention feature matrix and the synthetic position attention feature matrix, and aggregating the synthetic category attention feature matrix and the synthetic position attention feature matrix so that different matrices can exchange information, the synthesis method category and synthesis position of the face image are determined, thereby improving the efficiency and accuracy of identifying synthetic face images.

[0116] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a synthetic face image identification device provided in an embodiment of the present application. Figure 3 As shown in , the synthetic face image identification device 300 includes:

[0117] An image processing module 310 is used to perform image processing on a target face image to determine pixel clusters and feature clusters of the target face image;

[0118] An attention calculation module 320 is used to perform category attention calculation on each pixel set in the pixel cluster to determine a synthetic category attention feature matrix of each pixel set, and to perform position attention calculation on each feature set in the feature cluster to determine a synthetic position attention feature matrix of each feature set;

[0119] The synthesis determination module 330 is used to aggregate the multiple synthesis category attention feature matrices and the multiple synthesis position attention feature matrices to obtain the total attention feature vector of the target face image, perform full connection processing on the total attention feature vector, and determine the synthesis method category and synthesis position of the target face image.

[0120] Furthermore, when the image processing module 310 is used to perform image processing on the target face image and determine the pixel clusters and feature clusters of the target face image, the image processing module 310 is specifically used to:

[0121] Determining the pixel cluster in the target face image according to a pixel set under a target window centered on each pixel point;

[0122] Convolution processing, pooling processing and normalization processing are performed on each pixel set in the pixel cluster to determine a feature set corresponding to each pixel set, and the feature cluster is determined based on multiple feature sets.

[0123] Further, when the attention calculation module 320 is used to perform the category attention calculation on each pixel set in the pixel cluster to determine the synthetic category attention feature matrix of each pixel set in the pixel cluster, the attention calculation module 320 is specifically used to:

[0124] For each pixel set in the pixel cluster, based on the pixel set, the hyperparameter matrix and the pixel cluster, a query embedding vector corresponding to the pixel set is determined, and the pixel set, the hyperparameter matrix and the query embedding vector corresponding to the pixel set are processed based on the activation function to generate a synthetic category attention feature matrix for the pixel set.

[0125] Further, when the attention calculation module 320 is used to perform position attention calculation on each feature set in the feature cluster to determine the position attention feature matrix of each feature set, the attention calculation module 320 is specifically used to:

[0126] For each feature set in the feature cluster, the hyperparameter position matrix, the feature set and the parameter limit values ​​of the target window are processed based on the activation function to generate a synthetic position attention feature matrix of the feature set.

[0127] Furthermore, when the synthesis determination module 330 is used to perform full connection processing on the total attention feature vector to determine the synthesis method category and synthesis position of the target face image, the synthesis determination module 330 is specifically used to:

[0128] Splitting the total attention feature vector to determine a target synthesis category attention feature vector and a target synthesis position attention feature vector;

[0129] Performing result mapping processing on the target synthesis category attention feature vector based on a fully connected mapping function to map out the category of the synthesis method of the target face image;

[0130] The target synthesis position attention feature vector is mapped based on the fully connected mapping function to determine the synthesis position of the target face image.

[0131] Furthermore, the synthesis determination module 330 determines the total attention feature vector of the target face image by the following formula:

[0132]

[0133] Among them, S i is the synthetic category attention feature matrix, P i is the synthetic position attention feature matrix, d is the scaling factor, W is the learnable parameter matrix, m is the number of pixel sets, and A is the total attention feature vector.

[0134] The embodiment of the present application provides a synthetic face image identification device, the identification device includes: an image processing module, which is used to perform image processing on the target face image, and determine the pixel cluster and feature cluster of the target face image; an attention calculation module, which is used to perform category attention calculation on each pixel set in the pixel cluster, determine the synthetic category attention feature matrix of each pixel set, and perform position attention calculation on each feature set in the feature cluster to determine the synthetic position attention feature matrix of each feature set; a synthesis determination module, which is used to aggregate multiple synthetic category attention feature matrices and multiple synthetic position attention feature matrices to obtain the total attention feature vector of the target face image, perform full connection processing on the total attention feature vector, and determine the synthesis method category and synthesis position of the target face image. By using pixel clusters and feature clusters to determine the synthetic category attention feature matrix and the synthetic position attention feature matrix, and aggregating the synthetic category attention feature matrix and the synthetic position attention feature matrix so that different matrices can exchange information, the synthesis method category and synthesis position of the face image are determined, thereby improving the efficiency and accuracy of the identification of synthetic face images.

[0135] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0136] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 as well as Figure 2 The steps of the method for identifying the synthetic facial image in the method embodiment shown, and the specific implementation method can be found in the method embodiment, which will not be repeated here.

[0137] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 2 The steps of the method for identifying the synthetic facial image in the method embodiment shown, and the specific implementation method can be found in the method embodiment, which will not be repeated here.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0139] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0142] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0143] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for identifying synthetic face images, characterized in that: The identification method comprises: Performing image processing on the target face image to determine pixel clusters and feature clusters of the target face image; Performing category attention calculation on each pixel set in the pixel cluster to determine a synthetic category attention feature matrix of each pixel set, and performing position attention calculation on each feature set in the feature cluster to determine a synthetic position attention feature matrix of each feature set; Aggregating the plurality of synthetic category attention feature matrices and the plurality of synthetic position attention feature matrices to obtain a total attention feature vector of the target face image, performing full connection processing on the total attention feature vector to determine the synthetic method category and synthetic position of the target face image; The performing category attention calculation on each pixel set in the pixel cluster to determine a synthetic category attention feature matrix of each pixel set in the pixel cluster includes: For each pixel set in the pixel cluster, based on the pixel set, the hyperparameter matrix and the pixel cluster, determine a query embedding vector corresponding to the pixel set, process the pixel set, the hyperparameter matrix and the query embedding vector corresponding to the pixel set based on an activation function, and generate a synthetic category attention feature matrix of the pixel set; The performing position attention calculation on each feature set in the feature cluster to determine the position attention feature matrix of each feature set includes: For each feature set in the feature cluster, the hyperparameter position matrix, the feature set and the parameter limit values ​​of the target window are processed based on the activation function to generate a synthetic position attention feature matrix of the feature set.

2. The identification method according to claim 1, characterized in that: The performing image processing on the target face image to determine the pixel cluster and feature cluster of the target face image includes: Determining the pixel cluster in the target face image according to a pixel set under a target window centered on each pixel point; Convolution processing, pooling processing and normalization processing are performed on each pixel set in the pixel cluster to determine a feature set corresponding to each pixel set, and the feature cluster is determined based on multiple feature sets.

3. The identification method according to claim 1, characterized in that: The fully connected processing of the total attention feature vector to determine the synthesis method category and synthesis position of the target face image includes: Splitting the total attention feature vector to determine a target synthesis category attention feature vector and a target synthesis position attention feature vector; Performing result mapping processing on the target synthesis category attention feature vector based on a fully connected mapping function to map out the category of the synthesis method of the target face image; The target synthesis position attention feature vector is mapped based on the fully connected mapping function to determine the synthesis position of the target face image.

4. The identification method according to claim 1, characterized in that: The total attention feature vector of the target face image is determined by the following formula: Among them, S i is the synthetic category attention feature matrix, P i is the synthetic position attention feature matrix, d is the scaling factor, W is the learnable parameter matrix, m is the number of pixel sets, and A is the total attention feature vector.

5. A synthetic face image identification device, characterized in that: The identification device comprises: An image processing module, used to perform image processing on a target face image to determine a pixel cluster and a feature cluster of the target face image; An attention calculation module, used to perform category attention calculation on each pixel set in the pixel cluster to determine a synthetic category attention feature matrix of each pixel set, and to perform position attention calculation on each feature set in the feature cluster to determine a synthetic position attention feature matrix of each feature set; A synthesis determination module, used for aggregating a plurality of the synthesis category attention feature matrices and a plurality of the synthesis position attention feature matrices to obtain a total attention feature vector of the target face image, performing full connection processing on the total attention feature vector, and determining a synthesis method category and a synthesis position of the target face image; When the attention calculation module is used to perform category attention calculation on each pixel set in the pixel cluster to determine the synthetic category attention feature matrix of each pixel set in the pixel cluster, the attention calculation module is specifically used to: For each pixel set in the pixel cluster, based on the pixel set, the hyperparameter matrix and the pixel cluster, determine a query embedding vector corresponding to the pixel set, process the pixel set, the hyperparameter matrix and the query embedding vector corresponding to the pixel set based on an activation function, and generate a synthetic category attention feature matrix of the pixel set; When the attention calculation module is used to perform position attention calculation on each feature set in the feature cluster to determine the position attention feature matrix of each feature set, the attention calculation module is specifically used to: For each feature set in the feature cluster, the hyperparameter position matrix, the feature set and the parameter limit values ​​of the target window are processed based on the activation function to generate a synthetic position attention feature matrix of the feature set.

6. The identification device according to claim 5, characterized in that: When the image processing module is used to perform image processing on the target face image to determine the pixel cluster and feature cluster of the target face image, the image processing module is specifically used to: Determining the pixel cluster in the target face image according to a pixel set under a target window centered on each pixel point; Convolution processing, pooling processing and normalization processing are performed on each pixel set in the pixel cluster to determine a feature set corresponding to each pixel set, and the feature cluster is determined based on multiple feature sets.

7. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the method for identifying a synthetic facial image as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the synthetic facial image identification method according to any one of claims 1 to 4 are executed.

Images