Image detection methods, apparatus, electronic devices and readable storage media

Abstract

This application provides an image detection method, apparatus, electronic device, and readable storage medium to improve the accuracy of image detection. The method includes: performing a texture feature extraction operation on a target image to obtain an initial texture feature map of the target image; inputting the initial texture feature map of the target image into a first convolutional module to obtain texture features of a first convolutional layer; performing a first channel fusion operation on the texture features of the first convolutional layer to obtain a first fused texture feature; inputting the first fused texture feature into a second convolutional module to obtain texture features of a second convolutional layer; performing a second channel fusion operation on the texture features of the second convolutional layer to obtain a second fused texture feature; and performing a feature detection operation based on the second fused texture feature to determine whether the target image is a forged image; wherein the first convolutional module includes a first convolutional kernel, the second convolutional module includes a second convolutional kernel, and the first and second convolutional kernels have different sizes.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an image detection method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] The resurgence of artificial intelligence (AI) technology has greatly changed the way people live. Among them, the application of machine learning, especially convolutional neural networks (CNNs) in deep learning technology, has set off an unprecedented wave in the field of computer vision (CV).

[0003] On the one hand, facial recognition systems based on deep neural networks have made great strides, with applications including smart device unlocking, online payments, identity authentication, and smart office solutions. On the other hand, with the development of Generative Adversarial Networks (GANs), the quality of facial forgery samples generated through deepfake technology is becoming increasingly higher, and the forgery traces are becoming more concealed, thus posing a significant challenge to the field of facial security.

[0004] In related technologies, the problem of deep fake image detection, such as that of faces, is simply treated as a binary classification problem. This approach ignores the fact that deep fake image detection is essentially a fine-grained classification problem. Networks that focus more on fundamental features should be designed to extract more robust features, thereby improving the model's generalization performance. Therefore, how to efficiently and accurately detect deep fake images has become a pressing technical challenge. Summary of the Invention

[0005] This application provides an image detection method, apparatus, electronic device, and readable storage medium for efficiently and accurately detecting deep fake images.

[0006] In a first aspect, this application provides an image detection method, comprising the following steps:

[0007] A texture feature extraction operation is performed on the target image to obtain an initial texture feature map of the target image; wherein, the initial texture feature map of the target image is a feature map containing N channels;

[0008] The initial texture feature map of the target image is input into the first convolution module. The convolution operation result of the initial texture feature map is concatenated with the initial texture feature map to obtain the first convolution layer texture feature containing 2*N channels. The first channel fusion operation is performed on the first convolution layer texture feature to obtain the first fused texture feature containing N channels.

[0009] The first fused texture feature is input into the second convolution module. The convolution operation result of the first fused texture feature is concatenated with the first fused texture feature to obtain a second convolution layer texture feature containing 2*N channels. A second channel fusion operation is performed on the second convolution layer texture feature to obtain a second fused texture feature containing N channels. The first convolution module contains a first convolution kernel for performing the first convolution operation, and the second convolution module contains a second convolution kernel for performing the second convolution operation. The first convolution kernel and the second convolution kernel have different sizes.

[0010] A feature detection operation is performed based on the second fused texture feature to obtain a feature detection result, which is used to indicate whether the target image is a fake image.

[0011] Secondly, this application provides an image detection apparatus, comprising:

[0012] The feature extraction module is adapted to perform texture feature extraction operations on a target image to obtain an initial texture feature map of the target image; wherein, the initial texture feature map of the target image is a feature map containing N channels;

[0013] The first fusion module is adapted to input the initial texture feature map of the target image into the first convolution module, concatenate the convolution operation result of the initial texture feature map with the initial texture feature map to obtain the first convolution layer texture feature containing 2*N channels, and perform a first channel fusion operation on the first convolution layer texture feature to obtain the first fused texture feature containing N channels.

[0014] The second fusion module concatenates the convolution operation result of the first fused texture feature with the first fused texture feature to obtain a second convolutional layer texture feature containing 2*N channels, and performs a second channel fusion operation on the second convolutional layer texture feature to obtain a second fused texture feature containing N channels. The first convolution module contains a first convolution kernel for performing the first convolution operation, and the second convolution module contains a second convolution kernel for performing the second convolution operation. The first convolution kernel and the second convolution kernel have different sizes.

[0015] The detection module is adapted to perform feature detection operations based on the second fused texture features to obtain feature detection results, which are used to indicate whether the target image is a fake image.

[0016] Thirdly, this application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-described method.

[0017] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-described method when executed by a processor / processor core.

[0018] In the embodiments provided in this application, firstly, the initial texture feature map of the target image is input into a first convolutional module to obtain the texture features of the first convolutional layer, and a first channel fusion operation is performed on the texture features of the first convolutional layer to obtain the first fused texture features; then, the first fused texture features are input into a second convolutional module to obtain the texture features of the second convolutional layer, and a second channel fusion operation is performed on the texture features of the second convolutional layer to obtain the second fused texture features; finally, a feature detection operation is performed based on the second fused texture features to determine whether the target image is a forged image. In this embodiment, the size of the first convolutional kernel in the first convolutional module is different from the size of the second convolutional kernel in the second convolutional module. Correspondingly, the receptive fields of the texture features of the first convolutional layer and the texture features of the second convolutional layer are also different. Therefore, the basic features of the image can be reflected more accurately and comprehensively through texture features with different receptive fields, and the level of detail of the texture features with different receptive fields is different, thus resulting in better robustness. Furthermore, since this embodiment performs channel fusion operation on the texture features of the convolutional layer output by each convolutional module, it can more comprehensively reflect the relationship between each channel and its adjacent channels when the number of channels in the image is large, thereby further improving the accuracy of image detection.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the present application and form part of the specification. They are used together with the embodiments of the present application to explain the application and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed example embodiments described with reference to the accompanying drawings, in which:

[0021] Figure 1 A flowchart illustrating an image detection method provided in one embodiment of this application;

[0022] Figure 2 A schematic diagram of the multi-texture feature fusion process is shown;

[0023] Figure 3 A block diagram of an image detection device provided in this application embodiment;

[0024] Figure 4 This is a block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions of this application, exemplary embodiments of this application are described below in conjunction with the accompanying drawings, including various details of the embodiments of this application to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] Where there is no conflict, the various embodiments of this application and the features thereof may be combined with each other.

[0027] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0029] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this application, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0030] The image detection method according to the embodiments of this application can be executed by electronic devices such as terminal devices or servers. The terminal device can be an in-vehicle device, user device, mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant, handheld device, computing device, wearable device, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Specifically, the method can be implemented by a processor calling a computer program stored in memory.

[0031] In related technologies, the problem of deep fake image detection, such as that of faces, is simply treated as a binary classification problem, ignoring the fact that deep fake image detection is essentially a fine-grained classification problem, thus leading to low detection accuracy. To address this issue, this embodiment uses two convolutional modules with different kernel sizes for feature enhancement processing of texture features. This results in different receptive fields for the texture features of the first and second convolutional layers, allowing for a more accurate and comprehensive reflection of the image's fundamental features through texture features with different receptive fields. Furthermore, this embodiment performs channel fusion on the texture features output by each convolutional module, enabling a more comprehensive reflection of the relationship between each channel and its adjacent channels, even with a large number of image channels, thereby further improving image detection accuracy.

[0032] Figure 1 A flowchart illustrating an image detection method according to an embodiment of this application. (Refer to...) Figure 1 The method includes:

[0033] Step S110: Perform texture feature extraction on the target image to obtain the initial texture feature map of the target image; wherein, the initial texture feature map of the target image is a feature map containing N channels.

[0034] This involves detecting the target image to determine if it is a forged image. A forged image is an image that differs from the original image by adding forged information; for example, a forged image can be a deepfake image obtained through deepfake technology. In a specific example, a deepfake image mainly refers to a face image created using deepfake techniques.

[0035] Texture features are used to characterize the details such as texture contained in a target image. Deep fake images, after being processed, have their forgery traces diluted, and the appearance features revealed by different forgery techniques vary, making detection of deep fake images quite difficult. Deep feature learning schemes shift the focus from concrete features with details to abstract features with semantic information. By constructing multi-scale deep feature learning networks, they can consider not only individual objects and local contexts but also the relationships between objects. Deep networks have a large receptive field and strong semantic information representation ability, but low feature map resolution and weak geometric information representation ability (lacking spatial geometric feature details); low-level networks (also called shallow networks) have a small receptive field and strong geometric detail representation ability, although their semantic information representation ability is weak, they have high resolution. Considering the differences between deep and shallow networks, in this embodiment, texture features are extracted based on shallow networks.

[0036] For example, after inputting the target image into a feature extraction network, a series of feature maps of different sizes are obtained. A feature map downsampled by 3 times, i.e., 37*37 in size, is selected for texture feature extraction. Features with smaller downsampling factors (generally shallower layers) have smaller receptive fields, making them suitable for processing small targets and small-scale feature maps.

[0037] In one optional implementation, a texture feature extraction operation is performed on the target image using a first feature extraction method to obtain a first texture feature map; a texture feature extraction operation is then performed on the target image using a second feature extraction method to obtain a second texture feature map; the first and second texture feature maps are then merged to obtain an initial texture feature map. The first feature extraction method includes residual extraction, and the second feature extraction method includes a feature extraction method based on central difference convolution (CDC). By extracting texture features using two separate methods and then merging them (e.g., using the concat method), the final texture features obtained are more comprehensive and accurate.

[0038] The residual extraction method can be implemented as follows: First, the feature map obtained by downsampling the target image by a factor of 3 (referred to as feature map b2) is subjected to adaptive average pooling to reduce the feature dimension. For example, the 256x37x37 feature map b2 is transformed into a 256x19x19 feature map b3, where 256 is the number of channels and 37x37 is the image's width and height. Thus, after pooling, the number of pixels in the image is reduced from 37x37 to 19x19. Then, the nearest neighbor interpolation algorithm is performed on the 256x19x19 feature map b3 to obtain a 256x37x37 feature map b4. Therefore, through interpolation, the number of pixels in the image is expanded from 19x19 to 37x37. Finally, for the 256x37x37 feature map b2 and the 256x37x37 feature map b4, a sub-channel bitwise difference operation is performed to obtain the 256x37x37 residual texture feature (i.e., the first texture feature map).

[0039] Furthermore, the feature extraction method based on Central Difference Convolution (CDC) utilizes CDC operations, inspired by the Local Binary Pattern (LBP) algorithm, and tends to cluster the central gradient of sampled values. Therefore, in the process of texture feature extraction, fusing residual texture features and using additional texture features extracted by CDC can improve the accuracy of texture features, ensuring that the initial texture feature map of the target image more accurately reflects the texture of the target image. Figure 2 A schematic diagram of the multi-texture feature fusion process is shown, such as... Figure 2 As shown, a residual texture feature extraction operation is performed on the target image to obtain a first texture feature map of 256x37x37; a central difference convolution feature extraction operation is performed on the target image to obtain a second texture feature map of 256x37x37; then, a merging process is performed on the first and second texture feature maps to obtain an initial texture feature map of 256x37x37. Next, the initial texture feature map is input into a texture enhancement device (corresponding to the texture enhancement process in the figure) including at least two convolution modules. After processing by the texture enhancement device, a second fused texture feature map of 256x37x37 is obtained. The texture enhancement device includes the first convolution module and the second convolution module mentioned below, specifically used to perform steps S120 and S130 as described below. Therefore, in this embodiment, the number N is equal to 256.

[0040] Step S120: Input the initial texture feature map of the target image into the first convolution module, concatenate the result of the convolution operation of the initial texture feature map with the initial texture feature map to obtain the first convolution layer texture feature containing 2*N channels, and perform the first channel fusion operation on the first convolution layer texture feature to obtain the first fused texture feature containing N channels.

[0041] The first convolutional module includes a first convolutional kernel for performing the first convolutional operation. Accordingly, the first convolutional operation is performed on the initial texture feature map of the target image using the first convolutional kernel, and the texture features of the first convolutional layer are obtained based on the result of the convolutional operation. Therefore, the specific features of the texture features of the first convolutional layer depend on the size of the first convolutional kernel. Since the texture features of the first convolutional layer are related to the initial texture features... Figure 1 Similarly, since each convolutional layer has multiple channels, in order to accurately reflect the relationship between each channel and its adjacent channels, in this embodiment, a first channel fusion operation is further performed on the texture features of the first convolutional layer to obtain the first fused texture features. The channel fusion operation is used to fuse the data between the various channels, so that each channel can learn the data of its adjacent channels, thereby enabling the fused texture features to better reflect the features of the multiple channels. In an optional implementation, a preset convolution operation is performed on the texture features of the first convolutional layer to obtain the first fused texture features. Optionally, the preset convolution operation is a convolution operation performed using a 1x1 convolution kernel.

[0042] In this implementation, the number of channels in the convolution operation result of the initial texture feature map is the same as the number of channels in the initial texture feature map, which is also N channels. Correspondingly, after concatenating the convolution operation result of the initial texture feature map with the initial texture feature map, the number of channels doubles, resulting in the first convolutional layer texture feature containing 2*N channels. For example, when N equals 256, the first convolutional layer texture feature contains 512 channels. In another optional implementation, when inputting the initial texture feature map of the target image into the first convolution module to obtain the first convolutional layer texture feature, the specific implementation is as follows: First, the initial texture feature map of the target image is input into the first convolution module, and the convolution operation result of the initial texture feature map obtained after performing the first convolution operation through the first convolution kernel is used as the first texture feature; then, the first texture feature and the initial texture feature map are concatenated in groups to obtain the first convolutional layer texture feature containing multiple groups. Therefore, the texture features of the first convolutional layer are obtained by grouping and concatenating the input data (initial texture feature map) and the output data (first texture features) of the first convolutional module. Grouping and concatenation refers to further dividing the initial texture feature map and the first texture features into a predetermined number of groups during the concatenation process, and performing concatenation on each group separately. For example, if the initial texture feature map is divided into 4 groups and the first texture features are divided into 4 groups, then the data in the first group of the initial texture feature map is concatenated with the data in the first group of the first texture features to obtain the data of the first group in the first convolutional layer texture features; the data in the second group of the initial texture feature map is concatenated with the data in the second group of the first texture features to obtain the data of the second group in the first convolutional layer texture features, and so on. The reason for performing grouping is mainly to reduce the amount of data in each group, thereby facilitating computation. Furthermore, in this embodiment, the grouping method can also improve the effectiveness of the multi-head attention mechanism by assigning each attention map to a separate group when a multi-head attention mechanism is subsequently adopted. The grouping can be based on the location of the pixels within their respective regions. For example, pixels located in the upper left corner can be grouped into the first group, pixels in the upper right corner into the second group, pixels in the lower left corner into the third group, and pixels in the lower right corner into the fourth group.

[0043] For example, in a specific example, a first convolutional module with a kernel size of 3x3 processes the initial texture feature map to obtain the enhanced first texture feature. This enhanced first texture feature is then concatenated with the unenhanced initial texture feature map (e.g., using the concat method) to obtain the first convolutional layer texture feature. A 1x1 convolutional kernel is then used to fuse this first convolutional layer texture feature along the channel dimension, resulting in the first fused texture feature. This method effectively promotes information interaction between channels. Furthermore, the 1x1 convolution helps the model select features from the enhanced texture feature (i.e., the first texture feature) and the unenhanced texture feature (i.e., the initial texture feature map). During channel fusion, the number of channels output by the 1x1 convolutional kernel can be reduced by half compared to the number of input channels. Correspondingly, the first fused texture feature contains N channels, for example, 256. Therefore, it can be seen that by using the channel fusion operation, the number of channels can be reduced so that the number of channels in the first fused texture feature is the same as the number of channels in the initial texture feature map, which facilitates subsequent processing and enables better fusion of information between adjacent channels.

[0044] Step S130: Input the first fused texture feature into the second convolution module, concatenate the convolution operation result of the first fused texture feature with the first fused texture feature to obtain the second convolution layer texture feature containing 2*N channels, and perform the second channel fusion operation on the second convolution layer texture feature to obtain the second fused texture feature containing N channels.

[0045] The second convolution module includes a second convolution kernel for performing the second convolution operation. Accordingly, the second convolution operation is performed on the first fused texture features using the second convolution kernel, and the texture features of the second convolution layer are obtained based on the result of the convolution operation. Therefore, the specific features of the texture features of the second convolution layer depend on the size of the second convolution kernel.

[0046] Since the texture features of the second convolutional layer also have multiple channels, in order to accurately reflect the relationship between each channel and its adjacent channels, in this embodiment, a second channel fusion operation is further performed on the texture features of the second convolutional layer to obtain a second fused texture feature. The channel fusion operation is used to fuse the data between the various channels, so that each channel can learn the data of its adjacent channels, thereby enabling the fused texture feature to better reflect the features of multiple channels. In an optional implementation, a preset convolution operation is performed on the texture features of the second convolutional layer to obtain the second fused texture feature. Optionally, the preset convolution operation is a convolution operation performed using a 1x1 convolution kernel.

[0047] In this process, the convolution operation result of the first fused texture feature contains the same number of channels as the first fused texture feature itself, which is also N channels. Correspondingly, after concatenating the convolution operation result of the first fused texture feature with the first fused texture feature, the number of channels doubles, resulting in a second convolutional layer texture feature containing 2*N channels. For example, when N equals 256, the second convolutional layer texture feature contains 512 channels.

[0048] In another optional implementation, when the first fused texture feature is input into the second convolutional module to obtain the texture feature of the second convolutional layer, it is specifically implemented as follows: First, the first fused texture feature containing multiple groups is input into the second convolutional module. Based on the convolution operation result of the first fused texture feature obtained after the second convolution operation is performed by the second convolutional kernel, a second texture feature containing multiple groups is obtained. Then, the second texture feature and the first fused texture feature are subjected to group concatenation processing to obtain the texture feature of the second convolutional layer containing multiple groups; and the second fused texture feature contains multiple groups. Thus, the texture feature of the second convolutional layer is obtained by grouping and concatenating the input data (first fused texture feature) and the output data (second texture feature) of the second convolutional module. Group concatenation refers to further dividing the second texture feature and the first fused texture feature into a preset number of groups during the concatenation process, and performing concatenation processing on each group. The specific division and concatenation methods are similar to the previous step and will not be repeated here.

[0049] It should be noted that the second convolutional module contains a second convolutional kernel for performing the second convolutional operation, and the first and second convolutional kernels have different sizes. Performing convolution operations sequentially using convolutional modules of different sizes can improve the accuracy of image processing. Specifically, the receptive field of the data obtained after convolution operations using convolutional modules of different sizes is different; that is, the area reflected by each pixel after convolution is different. Therefore, by performing two convolution operations sequentially using convolutional kernels of different sizes, the receptive field of texture features can be changed to different degrees, thereby maximizing the enhancement of texture features and improving their discriminative power.

[0050] In one alternative implementation, the size of the first convolutional kernel is smaller than the size of the second convolutional kernel, and the receptive field of the texture features of the first convolutional layer is smaller than the receptive field of the texture features of the second convolutional layer. The inventors discovered during the implementation of this invention that by sequentially inputting the initial texture feature map into multiple convolutional modules arranged with kernel sizes ranging from small to large, the receptive field of the resulting fused texture features can be altered from small to large, thus better meeting the requirements of deep fake detection.

[0051] In another alternative implementation, the number of convolutional modules is not limited to two. For example, with three convolutional modules, after obtaining the second fused texture feature, it is further input into a third convolutional module to obtain the third convolutional layer texture feature. A third-channel fusion operation is then performed on the third convolutional layer texture feature to obtain the third fused texture feature. Correspondingly, in subsequent steps, feature detection operations are performed based on the third fused texture feature. Similarly, with four convolutional modules, after obtaining the third fused texture feature, it is further input into a fourth convolutional module to obtain the fourth convolutional layer texture feature. A fourth-channel fusion operation is then performed on the fourth convolutional layer texture feature to obtain the fourth fused texture feature. Correspondingly, in subsequent steps, feature detection operations are performed based on the fourth fused texture feature, and so on.

[0052] For example, in a specific example, the size of the first convolutional kernel in the first convolutional module is 3x3, the size of the second convolutional kernel in the second convolutional module is 5x5, and the size of the third convolutional kernel in the third convolutional module is 7x7.

[0053] Step S140: Perform feature detection operation based on the second fused texture features to obtain feature detection results. The feature detection results are used to indicate whether the target image is a fake image.

[0054] Since the second fused texture feature is extracted based on two texture feature extraction methods, and at least two convolutional processes with receptive fields increasing from small to large are performed using at least two convolutional modules of different sizes, and channel fusion processing is further performed after each convolutional process (i.e., cross-channel feature fusion is performed using 1x1 convolution) to better reflect the correlation between multiple channels, the second fused texture feature can better reflect the texture distribution characteristics of the target image. Therefore, feature detection operations can be performed based on the second fused texture feature, and the target image can be judged as a fake image (e.g., a deep fake image) based on the detection results.

[0055] Optionally, when performing feature detection based on the second fused texture features, a multi-head attention mechanism is used. The number of attention maps used in the multi-head attention mechanism is the same as the number of groups in the multiple groups, and each attention map corresponds to one group. Specifically, as mentioned above, during the concatenation process of the second texture features and the first fused texture features, the second texture features and the first fused texture features are further divided into a preset number of groups, and concatenation is performed on each group. Correspondingly, both the second convolutional layer texture features and the second fused texture features obtained after channel fusion contain multiple groups. By assigning a corresponding attention map to each group, different attention maps can pay attention to different group regions, thereby further improving the accuracy of the detection operation.

[0056] In one optional implementation, a second fused texture feature containing multiple groups is input into multiple attention modules contained in the image detection model; wherein, the multiple groups correspond one-to-one with the multiple attention modules; and, each attention module stores an attention map, and the attention maps in each attention module are different; the group detection result obtained by each attention module performing attention detection on the input group according to the internally stored attention map is obtained; and the feature detection result is determined based on the multiple group detection results obtained by the multiple attention modules.

[0057] The number of attention modules can be set as a hyperparameter, optimally selected based on ablation experiments, preferably 4. For example, assuming the second fused texture features contain four groups, a four-head attention mechanism is used to set four attention maps, each corresponding to one group. Group fusion processing essentially restricts the feature data of each group to be calculated only within that group, forcing each attention map to constrain the features of only one group, thus allowing the model to learn features with multiple different expressive capabilities. For instance, in a specific example, the feature map obtained by downsampling the target image by 5 times is used as the initial attention map. One or more convolution operations are performed on the initial attention map to obtain the attention map. In this embodiment, by controlling the number of output images from the convolution operation, four attention maps are output after the convolution operation, corresponding one-to-one with the four groups mentioned above. Then, the deep features downsampled by 5 times (i.e., the attention map) and the enhanced texture features (i.e., the second fused texture features) are fed together into a bilinear attention pooling module, combined, and the resulting features are sent to a classifier for detection. The specific operation performed by the bilinear attention pooling module is as follows: multiply the elements of the second fused texture feature with the elements of the attention map bitwise and then sum them. For example, for a 256*37*37 feature map (i.e., the second fused texture feature), perform the operation with a 37*37 attention map to obtain a 256-dimensional vector value, and then send the vector value to the classifier for classification and detection.

[0058] It should be noted that, since this embodiment generates a corresponding attention map for each group, ensuring that the attention map for each group is different, different regions can be focused on for different groups. For example, the group in the upper left corner may need more attention to the central region, while the group in the lower right corner may need more attention to the edge region. In short, setting different attention maps for different groups helps improve the targeting of group detection results and facilitates flexible setting of weight parameters in the corresponding attention map according to the characteristics of each group.

[0059] Additionally, it should be noted that all "convolution" operations mentioned in this embodiment refer to Central Difference Convolution (CDC) operations. Since CDC can enhance and enrich the representation of detailed features without increasing any parameters, it is more suitable for deep pseudo-feature detection. Under different hyperparameter settings, its representational capability encompasses both the representation of the original ordinary convolution and the central difference term. Furthermore, this embodiment uses a feature map obtained by downsampling the target image by 5 times as the attention map. Compared to related techniques that use a feature layer downsampled by 4 times as the attention map, this approach considers deeper feature maps and focuses on more abstract information, thereby improving accuracy.

[0060] In one optional implementation, the second fused texture feature is input into the image detection model, and the output of the image detection model is used to determine whether the target image is a forgery. The image detection model is the classifier mentioned above. The specific training process of the image detection model is as follows:

[0061] First, texture feature extraction is performed on the sample image to obtain an initial texture feature map; this initial texture feature map is a multi-channel feature map. Then, the initial texture feature map is input into the first convolutional module to obtain the texture features of the first convolutional layer. A first-channel fusion operation is performed on the texture features of the first convolutional layer to obtain the first fused texture features. The first fused texture features are then input into the second convolutional module to obtain the texture features of the second convolutional layer. A second-channel fusion operation is performed on the texture features of the second convolutional layer to obtain the second fused texture features. Training samples are generated based on the second fused texture features and the annotation results of the sample image (either a deep pseudo-image or a non-deep pseudo-image). The image detection model is then trained using these training samples. Therefore, the processing of the sample image is similar to that of the target image, and will not be elaborated further here. Furthermore, in the selection of the classifier loss function, AM-Softmax is used instead of ordinary Softmax, forcing the class boundaries to be more cohesive within classes and relatively separated between classes. Compared to Softmax, AM-Softmax adds a regularization term to Softmax, which adds a margin between the boundaries of different classes. As a result, the distance between classes is increased, thereby improving the robustness of the model.

[0062] Optionally, in a facial recognition system, it is first necessary to use liveness detection to determine whether the face image belongs to a live person, and then perform deep fake detection on the image to determine whether the image is a fake image. This is an essential part of modern facial recognition security authentication systems.

[0063] Accordingly, when the target image is a face image, a real or fake face image is input into the image detection model, and the model outputs the confidence level that the input image is a fake image. To improve the accuracy of image processing, a preprocessing operation is further performed on the target image before performing texture feature extraction:

[0064] First, face detection is performed on the image to be processed to obtain face images. The main function of face detection is to locate the position of faces in an image. In order to eliminate the interference of background and multiple faces, a face detection algorithm needs to be designed to accurately outline the outer boundary of the face with a rectangle, and the higher the accuracy, the better.

[0065] Then, keypoint detection is performed on the face image to obtain facial keypoints. The main function of facial keypoint detection is to provide key points of the face to complete subsequent face alignment. In related technologies, facial keypoint detection is used to detect 68 keypoints, while this invention can detect 98 keypoints of the face, making it more robust to blurred and rotated faces.

[0066] Finally, face alignment is performed on the face image based on facial key points to obtain the target image. The orientation and angle of the face can negatively impact subsequent deep fake detection; therefore, face alignment is essential to mitigate these effects. Face alignment refers to aligning any face to a standard face. This is equivalent to normalizing the face, ensuring that the features extracted by the subsequent model are independent of the position of facial features, focusing only on their texture and shape. Face alignment primarily involves image transformation to align the eyes, nose, and mouth to a preset, fixed position. This can be achieved by using two sets of corresponding points and obtaining a rotation matrix using OpenCV library functions. This matrix is ​​then used to perform an affine transformation on the face image to be aligned.

[0067] In one optional implementation, the deep fake detection module (i.e., image detection model) of the classification network is designed as an end-to-end model. The input is an aligned standard face image with a fixed size of 299*299 pixels and three channels of RGB. The output is the confidence score of whether the face image is a fake image. The higher the confidence score, the greater the likelihood of forgery. The classification threshold can be flexibly determined later according to business needs. The classification network consists of a backbone network and a network head, which will be described separately below.

[0068] The backbone network can use pre-trained feature extraction networks such as Xception. For ease of use, the intermediate modules of the feature extraction network are defined sequentially as b0…b10. Layer b2 is selected as the base layer for texture features, and layer b5 (or b6, b7) is selected as the base layer for the attention map. The texture feature dimension is 256*37*37, where 256 represents the number of channels and 37 represents the width and height of the feature map. The attention map dimension is 728*19*19, with each dimension having the same meaning as the texture feature. After obtaining the base texture features, two texture features are obtained using residual and CDC methods respectively. These two texture features are then concatenated by channel and fused using a 1x1 convolution. The final classifier is a two-layer fully connected neural network with 512 neurons in the intermediate layers and a final output dimension of 2, representing the probability that the target belongs to one of the two categories. After obtaining the confidence level that a face image belongs to the forgery type, it is then processed by subsequent logic.

[0069] In summary, the implementation method in this embodiment has at least the following advantages:

[0070] First, to obtain more robust features, this invention uses convolutional operations of various sizes (e.g., 3x3, 5x5, 7x7). Different sizes of convolutional operations create different receptive field sizes, somewhat resembling a spatial pyramid. Feature maps with different receptive field sizes are sequentially concatenated, and after each concatenation, a 1x1 convolution is used to fuse cross-channel information to integrate the features. This approach combines the advantages of residual connections and dense connections.

[0071] Secondly, after texture feature fusion and enhancement, it is concatenated and fused again with the originally extracted deep features. This is because this hierarchical network has the following characteristics: shallow features focus more on low-level features such as the object's shape, color, texture, and orientation, while deep features focus more on the object's semantic features, such as category. Borrowing the idea of ​​fusing shallow and deep features from the field of object detection, this operation can obtain richer semantic features and improve model performance.

[0072] Furthermore, during the network training phase, the AM-Softmax loss function was used to reduce intra-class differences and increase inter-class differences, while Softmax activation was directly used during the testing phase. This invention combines residual and central difference convolution methods to extract texture features, replacing all subsequent convolutions with CDC and designing a densely connected neural network to enhance texture features. Finally, it combines deep and shallow layer features for classification, making the extracted features more robust and effective. Combined with attention map analysis, the model pays more attention to the fake regions of fake faces, thereby improving model accuracy.

[0073] It is understood that the various method embodiments mentioned above in this application can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this application will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0074] In addition, this application also provides an image detection device, an electronic device, and a computer-readable storage medium, all of which can be used to implement any of the image detection methods provided in this application. The corresponding technical solutions and descriptions are described in the corresponding descriptions in the method section and will not be repeated here.

[0075] Figure 3 A block diagram of an image detection device 30 provided in this application embodiment.

[0076] Reference Figure 3 This application provides an image detection device 30, which includes:

[0077] The feature extraction module 31 is adapted to perform texture feature extraction operations on the target image to obtain an initial texture feature map of the target image; wherein, the initial texture feature map of the target image is a feature map containing N channels;

[0078] The first fusion module 32 is adapted to input the initial texture feature map of the target image into the first convolution module, concatenate the convolution operation result of the initial texture feature map with the initial texture feature map to obtain the first convolution layer texture feature containing 2*N channels, and perform a first channel fusion operation on the first convolution layer texture feature to obtain the first fused texture feature containing N channels.

[0079] The second fusion module 33 is adapted to input the first fused texture feature into the second convolution module, concatenate the convolution operation result of the first fused texture feature with the first fused texture feature to obtain a second convolution layer texture feature containing 2*N channels, and perform a second channel fusion operation on the second convolution layer texture feature to obtain a second fused texture feature containing N channels. The first convolution module contains a first convolution kernel for performing the first convolution operation, the second convolution module contains a second convolution kernel for performing the second convolution operation, and the first convolution kernel and the second convolution kernel have different sizes.

[0080] The detection module 34 is adapted to perform feature detection operations based on the second fused texture features to obtain feature detection results, which are used to indicate whether the target image is a fake image.

[0081] Optionally, the feature extraction module is specifically adapted to: perform texture feature extraction operation on the target image using a first feature extraction method to obtain a first texture feature map; perform texture feature extraction operation on the target image using a second feature extraction method to obtain a second texture feature map; and perform merging processing on the first texture feature map and the second texture feature map to obtain the initial texture feature map.

[0082] Optionally, the first feature extraction method is a residual extraction method, and the second feature extraction method is a feature extraction method based on central difference convolution.

[0083] Optionally, the size of the first convolutional kernel is smaller than the size of the second convolutional kernel, and the receptive field of the texture features of the first convolutional layer is smaller than the receptive field of the texture features of the second convolutional layer.

[0084] Optionally, the first fusion module is specifically adapted to: perform a preset convolution operation on the texture features of the first convolutional layer to obtain the first fused texture features; the second fusion module is specifically adapted to: perform a preset convolution operation on the texture features of the second convolutional layer to obtain the second fused texture features; wherein, the preset convolution operation is a convolution operation performed through a 1x1 convolution kernel.

[0085] Optionally, the first fusion module is specifically adapted to: input the initial texture feature map of the target image into the first convolution module to obtain the first texture feature after performing a first convolution operation through the first convolution kernel; perform grouping and concatenation processing on the first texture feature and the initial texture feature map to obtain a first convolutional layer texture feature containing multiple groups; and the second fusion module is specifically adapted to: input the first fused texture feature into the second convolution module to obtain the second texture feature after performing a second convolution operation through the second convolution kernel; perform grouping and concatenation processing on the second texture feature and the first fused texture feature to obtain a first convolutional layer texture feature containing multiple groups; and the detection module is specifically adapted to: perform feature detection operation on the second fused texture feature through a multi-head attention mechanism; wherein the number of attention maps used in the multi-head attention mechanism is the same as the number of groups in the multiple groups, and each attention map corresponds to one group.

[0086] Optionally, the feature extraction module is further adapted to: perform face detection processing on the image to be processed to obtain a face image; perform key point detection processing on the face image to obtain face key points; and perform face alignment processing on the face image based on the face key points to obtain the target image.

[0087] Optionally, the detection module is specifically adapted to: input the second fused texture features into an image detection model, and determine whether the target image is a fake image based on the output of the image detection model.

[0088] Figure 4 A block diagram of an electronic device is provided in this application embodiment.

[0089] Reference Figure 4 This application provides an electronic device, which includes: at least one processor 501; at least one memory 502; and one or more I / O interfaces 503 connected between the processor 501 and the memory 502; wherein the memory 502 stores one or more computer programs that can be executed by the at least one processor 501, and the one or more computer programs are executed by the at least one processor 501 to perform the above-described image detection method.

[0090] This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor / processor core, implements the data migration method described above. The computer-readable storage medium can be volatile or non-volatile.

[0091] This application also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described data migration method.

[0092] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0093] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0094] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0095] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (SA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing the status information of the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0096] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0097] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0098] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0099] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0101] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this application as set forth by the appended claims.

Claims

1. An image detection method, characterized in that, include: A texture feature extraction operation is performed on the target image to obtain an initial texture feature map of the target image; wherein, the initial texture feature map of the target image is a feature map containing N channels; The initial texture feature map of the target image is input into the first convolution module. The convolution operation result obtained after the first convolution operation of the initial texture feature map is concatenated with the initial texture feature map to obtain the first convolution layer texture feature containing 2 x N channels. The first channel fusion operation is performed on the first convolution layer texture feature to obtain the first fused texture feature containing N channels. The first fused texture feature is input into the second convolution module. The convolution operation result obtained after the second convolution operation of the first fused texture feature is concatenated with the first fused texture feature to obtain a second convolution layer texture feature containing 2 x N channels. A second channel fusion operation is performed on the second convolution layer texture feature to obtain a second fused texture feature containing N channels. The first convolution module contains a first convolution kernel for performing the first convolution operation, and the second convolution module contains a second convolution kernel for performing the second convolution operation. The first convolution kernel and the second convolution kernel have different sizes. A feature detection operation is performed based on the second fused texture feature to obtain a feature detection result, which is used to indicate whether the target image is a fake image.

2. The method according to claim 1, characterized in that, The step of performing texture feature extraction on the target image to obtain the initial texture feature map of the target image includes: A first texture feature map is obtained by performing texture feature extraction on the target image using a first feature extraction method. A second texture feature map is obtained by performing texture feature extraction on the target image using a second feature extraction method. A merging process is performed on the first texture feature map and the second texture feature map to obtain the initial texture feature map.

3. The method according to claim 2, characterized in that, The first feature extraction method is the residual extraction method, and the second feature extraction method is the feature extraction method based on central difference convolution.

4. The method according to claim 1, characterized in that, The size of the first convolutional kernel is smaller than the size of the second convolutional kernel, and the receptive field of the texture features of the first convolutional layer is smaller than the receptive field of the texture features of the second convolutional layer.

5. The method according to claim 1, characterized in that, The first channel fusion operation and the second channel fusion operation are preset convolution operations. The first fused texture feature is obtained by performing the preset convolution operation on the texture feature of the first convolutional layer, and the second fused texture feature is obtained by performing the preset convolution operation on the texture feature of the second convolutional layer. The preset convolution operation is a convolution operation performed by a 1 x 1 convolution kernel and is used for channel fusion processing.

6. The method according to claim 1, characterized in that, The process of inputting the initial texture feature map of the target image into the first convolution module, and concatenating the convolution operation result obtained after the first convolution operation of the initial texture feature map with the initial texture feature map to obtain the first convolution layer texture feature containing 2 x N channels includes: The initial texture feature map of the target image is input into the first convolution module, and the convolution operation result of the initial texture feature map obtained after performing the first convolution operation through the first convolution kernel is used as the first texture feature; The first texture feature and the initial texture feature map are divided into multiple groups and group concatenation is performed to obtain a first convolutional layer texture feature containing multiple groups; and the first fused texture feature contains multiple groups. The step of inputting the first fused texture feature into the second convolution module, and concatenating the convolution operation result obtained after the second convolution operation of the first fused texture feature with the first fused texture feature to obtain a second convolutional layer texture feature containing 2 x N channels includes: The first fused texture feature containing multiple groups is input into the second convolution module. Based on the convolution operation result of the first fused texture feature obtained after the second convolution operation is performed by the second convolution kernel, a second texture feature containing multiple groups is obtained. The second texture feature and the first fused texture feature are subjected to group concatenation processing to obtain a second convolution layer texture feature containing multiple groups. The second fused texture feature contains multiple groups.

7. The method according to claim 6, characterized in that, The feature detection operation based on the second fused texture features to obtain the feature detection result includes: The second fused texture features containing multiple groups are input into multiple attention modules contained in the image detection model; wherein, the multiple groups correspond one-to-one with the multiple attention modules; and, each attention module stores an attention map, and the attention maps in each attention module are different. Obtain the group detection results obtained by each attention module after performing attention detection on the input groups based on the internally stored attention map; The feature detection result is determined based on multiple group detection results obtained from multiple attention modules.

8. An image detection device, characterized in that, include: The feature extraction module is adapted to perform texture feature extraction operations on a target image to obtain an initial texture feature map of the target image; wherein, the initial texture feature map of the target image is a feature map containing N channels; The first fusion module is adapted to input the initial texture feature map of the target image into the first convolution module, concatenate the convolution operation result obtained after the first convolution operation of the initial texture feature map with the initial texture feature map to obtain the first convolution layer texture feature containing 2 x N channels, and perform a first channel fusion operation on the first convolution layer texture feature to obtain the first fused texture feature containing N channels. The second fusion module concatenates the convolution result obtained after the first fused texture feature undergoes a second convolution operation with the first fused texture feature to obtain a second convolutional layer texture feature containing 2 x N channels. Then, it performs a second channel fusion operation on the second convolutional layer texture feature to obtain a second fused texture feature containing N channels. The first convolution module contains a first convolution kernel for performing the first convolution operation, and the second convolution module contains a second convolution kernel for performing the second convolution operation. The first convolution kernel and the second convolution kernel have different sizes. The detection module is adapted to perform feature detection operations based on the second fused texture features to obtain feature detection results, which are used to indicate whether the target image is a fake image.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more of the computer programs being executed by the at least one processor to enable the at least one processor to perform the image detection method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image detection method as described in any one of claims 1-7.

Images