Methods, apparatus, devices, media, and products for image detection

Abstract

According to embodiments of the present disclosure, a method, apparatus, device, medium and product for image detection are provided. The method comprises: determining a first feature of a target image by performing a transformation from a spatial domain to a frequency domain on the target image, the first feature comprising at least one feature vector corresponding to at least one frequency channel respectively; determining a first detection result about whether the target image is modified based on the first feature; determining at least one second detection result about whether the target image is modified based on at least one second feature of the target image, a dimension of the at least one second feature being different from a dimension of the first feature; and determining a target detection result about whether the target image is modified based on the first detection result and the at least one second detection result. In this way, the detection performance can be further optimized while enhancing the accuracy and robustness of image detection.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatus, devices, and computer-readable storage media for image detection. Background Technology

[0002] With the rapid development of internet technology, images of all kinds have become an important medium for information dissemination, entertainment, and social communication. However, with the popularization of image editing tools and machine learning technologies, the application of image manipulation techniques has become increasingly widespread. Manipulated images may be used for the spread of false information, misleading reporting, and other scenarios with security risks. This places higher demands on the detection of image authenticity. Summary of the Invention

[0003] In a first aspect of this disclosure, an image detection method is provided. The method includes: determining a first feature of the target image by performing a transformation from a spatial domain to a frequency domain on the target image, the first feature including at least one feature vector corresponding to at least one frequency channel; determining a first detection result regarding whether the target image has been modified based on the first feature; determining at least one second detection result regarding whether the target image has been modified based on at least one second feature of the target image, the dimension of the at least one second feature being different from the dimension of the first feature; and determining a target detection result regarding whether the target image has been modified based on the first detection result and the at least one second detection result.

[0004] In a second aspect of this disclosure, an image detection apparatus is provided. The apparatus includes: a feature determination module configured to determine a first feature of a target image by performing a spatial-to-frequency domain transformation on the target image, the first feature including at least one feature vector corresponding to at least one frequency channel; a first detection result determination module configured to determine a first detection result regarding whether the target image has been modified based on the first feature; a second detection result determination module configured to determine at least one second detection result regarding whether the target image has been modified based on at least one second feature of the target image, the dimension of the at least one second feature being different from the dimension of the first feature; and a target detection result determination module configured to determine a target detection result regarding whether the target image has been modified based on the first detection result and the at least one second detection result.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0008] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0010] Figure 2 A schematic diagram of the architecture of a system for image detection according to some embodiments of the present disclosure is shown;

[0011] Figure 3A A schematic diagram of an example process for determining a first feature according to some embodiments of the present disclosure is shown;

[0012] Figure 3B A schematic diagram of the architecture of a first detection model according to some embodiments of the present disclosure is shown;

[0013] Figure 3C A schematic diagram of the architecture of a fusion subsystem according to some embodiments of the present disclosure is shown;

[0014] Figure 4 A flowchart illustrating an image detection process according to some embodiments of the present disclosure is shown;

[0015] Figure 5 A schematic structural block diagram of an apparatus for image detection according to certain embodiments of the present disclosure is shown; and

[0016] Figure 6 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0019] As used in this paper, the term "model" refers to a system that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this paper, "model" may also be referred to as a "machine learning model," "machine learning network," "neural network," or simply "network," and these terms are used interchangeably.

[0020] A neural network is a machine learning network based on deep learning. A neural network processes input and provides a corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications often include many hidden layers, thus increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer is provided as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output serves as the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each node processing the input from the layer above.

[0021] Machine learning typically comprises three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating its parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0022] In this document, unless explicitly stated otherwise, performing a step in response to A does not mean that the step is performed immediately after A, but may include one or more intermediate steps.

[0023] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0025] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0026] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, such as a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0028] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. Figure 1 In environment 100, it is desirable to train and use a machine learning model (i.e., model 130) configured for various application environments. For example, in the case that model 130 is an image detection model, it is possible to determine, based on the input image, whether the target detection result has been modified, etc.

[0029] like Figure 1 As shown, environment 100 includes model training system 150 and model application system 160. Figure 1 The upper part illustrates the model training phase, and the lower part illustrates the model application phase. Before training, the parameter values ​​of model 130 can have initial values ​​or pre-trained parameter values ​​obtained through a pre-training process. Model 130 can be trained via forward and backward propagation, during which the parameter values ​​of model 130 can be updated and adjusted. After training is complete, model 130' is obtained. At this point, the parameter values ​​of model 130' have been updated, and based on the updated parameter values, model 130 can be used to implement the image detection task in the model application phase.

[0030] During the model training phase, the model 130 can be trained using a training sample set 110 comprising multiple training samples 112 and a model training system 150. The training samples 112, including model input 120 and model output 122, can be used to train the model 130. Specifically, the training process can be performed iteratively using a large number of training samples. After training is complete, the model 130 can include knowledge about the task to be processed.

[0031] During the model application phase, model 130' (which at this point has the trained parameter values) can be used to perform the corresponding task. Specifically, model application system 160 can receive model input 142 and output the corresponding model output 144. For example, image detection system 160 determines the target detection result 144 regarding whether the input image has been modified based on the input image 142.

[0032] exist Figure 1 In this context, the model application system 160 may include any computing system with computing capabilities, such as various computing devices / systems, terminal devices, servers, etc. Terminal devices may involve any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. Servers include, but are not limited to, mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0033] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0034] As briefly mentioned earlier, with the rapid development of internet technology, various images play an irreplaceable role in fields such as social media, news reporting, and advertising. However, with the widespread use of image editing tools and machine learning technologies, the barrier to digital image tampering has been significantly lowered. Various image content tampering behaviors are becoming increasingly common. Tampering techniques include image content forgery, splicing, cloning, partial modification, and face swapping using machine learning. Tampered images may be used for spreading false information, misleading reporting, and other scenarios with security risks. To address these issues, image detection technologies that identify whether images have been modified have emerged. Several typical solutions for detecting whether images have been modified are described below.

[0035] A typical solution involves feature-based image detection. Specifically, it detects image manipulation traces by extracting specific physical or statistical properties (such as texture, edges, color distribution, etc.). However, these methods rely on manually designed features and classification using traditional machine learning models. Consequently, they lack accuracy and robustness when dealing with complex tampering and large-scale image processing.

[0036] Another typical solution involves deep learning-based image detection. Specifically, a deep learning model is trained on a large amount of labeled data. The model can automatically learn high-level features in images. However, deep learning models are generally better at learning semantic features in images, such as those related to the category or shape of objects. Tamper detection is more concerned with inconsistencies or forgery traces in images, which are unrelated to the semantic features of the objects themselves. Furthermore, this type of solution also faces the problems of high computational cost and a high dependence on the quality of training data.

[0037] In view of the above, according to embodiments of this disclosure, an improved scheme for image detection is proposed to at least partially solve one or more of the above-mentioned problems. The scheme includes determining a first feature of a target image by performing a transformation from the spatial domain to the frequency domain on the target image, the first feature including at least one feature vector corresponding to at least one frequency channel. Then, based on the first feature, a first detection result regarding whether the target image has been modified is determined. Based on at least one second feature of the target image, at least one second detection result regarding whether the target image has been modified is determined, the dimension of the at least one second feature being different from the dimension of the first feature. Based on the first detection result and the at least one second detection result, a target detection result regarding whether the target image has been modified is determined.

[0038] In the embodiments of this disclosure, multidimensional features in the frequency and spatial domains can capture more complex and diverse details of image modifications (e.g., image tampering). This enhances the detection capability for complex modification scenarios, improving the accuracy and robustness of image detection. Furthermore, by extracting key frequency domain channel information during frequency domain transformation and preserving the frequency features of the modified region, detection performance can be further optimized while enhancing the detection capability for low-quality images.

[0039] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0040] Figure 2 A schematic diagram of the architecture of a system 200 for image detection according to some embodiments of the present disclosure is shown. Figure 2 System 200, for example, can be implemented as Figure 1 The model application system 160 is implemented or deployed in the model application system 160. For example... Figure 2 As shown, system 200 may include a preprocessing subsystem 210. The preprocessing subsystem 210 is used to acquire multi-dimensional features of the target image 210. Specifically, by performing feature extraction operations on the target image 210, the preprocessing subsystem 210 can determine features of various dimensions, providing multi-dimensional information support for subsequent detection.

[0041] In some embodiments, the preprocessing subsystem 210 determines a first feature 211 of the target image 210 by performing a transformation from the spatial domain to the frequency domain on the target image 210. The first feature 211 includes at least one feature vector corresponding to at least one frequency channel. The first feature 211 is determined based on the frequency domain information of the image. Each frequency channel may correspond to a frequency or a frequency band. Therefore, the data of each frequency channel reflects the variations at different frequencies.

[0042] Figure 3A A schematic diagram of an example process 310 for determining a first feature according to some embodiments of the present disclosure is shown. Process 310 may be implemented by a preprocessing subsystem 210.

[0043] refer to Figure 3AFirst, the target image 201 is divided into multiple image blocks. In block 311, the preprocessing subsystem 210 determines the corresponding frequency domain information 304 of the multiple image blocks by performing a spatial-to-frequency domain transformation on each image block. The frequency domain information of each image block includes the corresponding transform coefficients of that image block in multiple frequency channels. The number of these frequency channels is related to the size of the image block; for example, the number of these frequency channels can be equal to the number of pixels included in the image block. By statistically analyzing the frequency domain information (i.e., transform coefficients) of each image block in each frequency channel, the corresponding coefficient histogram is obtained as the feature vector of that channel. The distribution pattern of the histogram can reflect frequency characteristics such as the difference in compression times that may exist between tampered and untampered areas in the image, thereby helping to capture changes in the modified areas of the image.

[0044] In some embodiments, the transformation from the spatial domain to the frequency domain may include a Discrete Cosine Transform (DCT), with transform coefficients including DCT coefficients. As an example, the original dimensions of the target image 201 are height H × width W × number of color channels 3. Using 8×8 pixels as the basic unit, the target image 201 is divided into multiple 8×8 image blocks. A DCT transform is performed on each 8×8 image block, generating a DCT coefficient matrix of H / 8×W / 8×64, where 64 represents the number of frequency channels corresponding to the frequency domain components after the DCT transform.

[0045] Traditionally, DCT transform methods for extracting frequency domain features from images typically require several steps (e.g., 4 to 5 steps) to ultimately obtain a two-dimensional heatmap as the output feature. However, these multiple processing and algorithmic steps not only increase the computational burden of image detection but also lead to information loss. Therefore, in some embodiments, information from key frequency channels can be extracted during the frequency domain transform to obtain multiple one-dimensional feature vectors, thereby further optimizing detection performance while preserving the frequency features of the modified region.

[0046] Continue to refer to Figure 3AIn box 312, the preprocessing subsystem 210 statistically analyzes the corresponding frequency domain information of multiple image blocks according to multiple frequency channels to determine feature vectors 306 corresponding to at least one frequency channel (e.g., n frequency channels, where n is a positive integer greater than or equal to 1). The at least one frequency channel involved in the first feature 211 can be at least a portion of the aforementioned multiple frequency channels. For example, the first n channels are selected from 64 frequency channels, sorted from low to high frequency, and the corresponding DCT coefficients are statistically analyzed to determine n one-dimensional feature vectors. Here, n can be less than or equal to 64. Since low-frequency channels not only contain the main information of the image but also more clearly demonstrate the histogram quantization effect caused by multiple image compressions, selecting the first n low-frequency channels helps improve the effectiveness of feature extraction.

[0047] In some embodiments, because the DCT coefficients of different images have different value ranges, the length of the resulting one-dimensional feature vector may vary after calculation. To facilitate processing by the detection model, the length of these vectors needs to be standardized to a fixed value. For example, if the vector length is less than 1024, it can be padded with zeros at both ends. If it exceeds 1024, it can be truncated according to specific needs. Note that the specific values ​​of the vector lengths described herein are merely exemplary and are not intended to impose any limitations.

[0048] In some embodiments, at least one frequency channel is selected from multiple frequency channels. For example, from 64 DCT channels, the DCT coefficients of channels 2 through 15 are selected as at least one frequency channel corresponding to at least one feature vector included in the first feature. These 14 channels contain low-frequency and partially mid-frequency information of the image, effectively reflecting detail anomalies caused by image compression and modification. The preprocessing subsystem 210 then arranges the DCT coefficients corresponding to the second frequency channel in all image blocks into a histogram, arranges the DCT coefficients corresponding to the third frequency channel in all image blocks into another histogram, and so on. Ultimately, 14 histograms are obtained, each corresponding to one of the selected second through 15 DCT channels. Fourteen one-dimensional feature vectors are formed based on these 14 histograms. The selected frequency channels described in the above embodiments are merely exemplary and are not intended to be limiting. In other embodiments, other suitable one or more frequency channels may also be selected as at least one frequency channel for determining the first feature.

[0049] The extracted features are n one-dimensional feature vectors, rather than traditional two-dimensional feature heatmaps. This allows the first feature to include key details of image modifications while significantly reducing computational load. In this way, detection performance can be further optimized while enhancing the accuracy and robustness of image detection.

[0050] Return to reference Figure 2 The preprocessing subsystem 210 can also determine at least one second feature with a dimension different from the first feature 211 based on the target image 201. For example, Figure 2 Second features 212-1, 212-2, ..., 212-N are shown, which may be collectively referred to as second features or individually, where N is a positive integer. In some embodiments, at least one second feature 212 may include feature 212-1 represented by a plurality of color channels.

[0051] As an example, feature 212-1, represented by multiple color channels, can include the RGB features of the target image 201. RGB features are the original color distribution features of an image in pixel space, directly reflecting the basic visual information of the image. RGB features are suitable for general image processing tasks and can capture the overall structure and local texture information of an image.

[0052] In some embodiments, at least one second feature 212 may further include a feature 212-2 represented by a luminance channel and at least one chrominance channel. For example, a color space transformation is performed on a target image in RGB format with an original size of H×W×3 to obtain a YCbCr feature with a size of H×W×3, where Y represents the luminance information of the image, Cb represents the deviation of the blue channel relative to luminance, and Cr represents the deviation of the red channel relative to luminance. The YCbCr feature can explicitly separate color and luminance information, and is very effective for detecting luminance changes or color inconsistencies caused by image modifications. Therefore, feature 212-2 represented by a luminance channel and at least one chrominance channel is particularly suitable for scenarios that detect modified areas (such as luminance adjustment, reflection modification, etc.) caused by abnormal luminance or color distribution.

[0053] In some embodiments, at least one second feature 212 may further include features based on image noise analysis. Features based on image noise analysis include, for example, Steganalysis Rich Model (SRM) features. SRM is an algorithmic model used to detect hidden information (such as information hiding or steganography) in an image. It is widely used in the field of image modification detection, especially in scenarios where images have been modified to insert forged data. SRM can use the statistical information of the image to determine whether there is potential modification or information hiding based on the extraction and analysis of multiple features.

[0054] As an example, noise residuals in an H×W×3 RGB format target image are extracted by inputting it into a specific filter (e.g., a 5×5×3 filter). Through filtering, a set of SRM features representing the image noise residual information is generated, with the same size as the original target image. SRM features reflect noise patterns and discontinuities in local regions of the image. Based on SRM features, anomalous noise characteristics caused by image modification can be captured.

[0055] The above describes several examples of the second feature 212. It should be understood that in other embodiments, the second feature 212 may also include more, fewer, or any other features suitable for image detection.

[0056] In the embodiments of this disclosure, by extracting dimensional features of the target image, information about the image at multiple levels, such as color, brightness, texture, and high-frequency noise, is comprehensively captured. This effectively improves the diversity and discriminative power of image feature representation, providing higher-quality input for subsequent detection models.

[0057] Return to reference Figure 2 System 200 may include a modification detection subsystem 220. In the modification detection subsystem 220, multi-dimensional features determined based on the target image 201 are respectively input into one or more detection models to determine the corresponding detection results. In some embodiments, the modification detection subsystem 220 may determine a first detection result 221 regarding whether the target image has been modified based on a first feature 211. For example, a first detection model based on a one-dimensional convolution operation may be used to determine the first detection result based on the first feature.

[0058] As an example, Figure 3B A schematic diagram of the architecture 320 of a first detection model according to some embodiments of the present disclosure is shown. Figure 3B As shown, the first feature 211 obtained through the transformation operation from the spatial domain to the frequency domain is input into the first detection model. The first feature 211 includes n one-dimensional feature vectors corresponding to n frequency channels. Therefore, to adapt to the one-dimensional input, the first detection model includes several one-dimensional convolutional layers 322 (e.g., 1×3 conv1d convolutional layers) to perform one-dimensional convolution operations on the first feature 211. The one-dimensional convolution operation can effectively capture the dependencies between features in the one-dimensional feature vector and further reduce the computational complexity.

[0059] As an example, a first feature of size 1×14×1024 is input into the first detection model. It is first passed through a 1x3 one-dimensional convolutional layer (32 channels), resulting in output data of size 32×14×512. This is followed by multiple 1x3 convolutional operations, progressively adjusting the number of channels and the dimensionality of the data, resulting in output data of size 256×14×32. Note that the dimensions and specific values ​​of the first feature and the one-dimensional convolutional layer described here are merely illustrative and are not intended to be any limitation.

[0060] Continue to refer to Figure 3B The feature representation obtained after one-dimensional convolution is then fed into pooling layer 323. In pooling layer 323, average pooling (14×1 average pooling) is performed on the 256×14×32 feature representation, resulting in a pooling result of size 256×14×1. This pooling result is then flattened into a 256×14 vector and fed into fully connected layer 324.

[0061] In the fully connected layer 324, a feature representation of size 1×2 is obtained through further feature learning. Then, the Softmax activation function 325 maps the 1×2 feature representation to probability scores (e.g., the probabilities of the "modified" and "unmodified" categories). Finally, the modification detection subsystem 220 can output the probability scores mapped by the activation function 325 as the first detection result 221.

[0062] In the embodiments of this disclosure, latent modification information in the first feature is extracted through layer-by-layer convolution, pooling, and fully connected operations, and the final detection result is output through a classifier. This design can effectively capture hidden modification traces such as multiple compressions in the frequency domain features, improving detection accuracy. Furthermore, using one-dimensional convolution operations instead of more complex two-dimensional or three-dimensional convolutions reduces computational load and helps improve detection efficiency.

[0063] Continue to refer to Figure 2 The modified detection subsystem 220 can also utilize at least one second target model corresponding to different types of second features 212 to determine at least one second detection result 222. Specifically, the modified detection subsystem 220 can, for any suitable type of second feature 212, determine one or more detection results corresponding to that second feature 212 based on one or more second detection models. Then, the one or more detection results are determined as part of at least one second detection result 222.

[0064] As an example, features such as RGB, YCbCr, and SRM can be input into one or more detection models (e.g., specialized detection networks, lightweight networks, or convolutional neural networks). For each different type of feature input, the detection model outputs a detection result representing the likelihood of forgery and modification of the image across various characteristics such as color space, brightness space, and high-frequency features. If multiple detection models are used for a single feature (e.g., analyzing features extracted by different SRM filters or different channels of the YCbCr feature), each model can output an independent score, and the total score for that type of feature is calculated by aggregation.

[0065] In some embodiments, the second detection model may include a model based on depthwise separable convolutions. Models based on depthwise separable convolutions may include, for example, any suitable depthwise separable convolutional network. By employing depthwise separable convolutions, which divide convolution into two steps—spatial convolution and channel convolution—the number of parameters and computational complexity is greatly reduced, while retaining high feature extraction capabilities.

[0066] Alternatively or additionally, the second detection model may include a model based on efficient convolutions. Efficient convolution-based models include, for example, efficient convolutional networks with any suitable structure. An optimized network architecture, obtained through automated search, employs a compound scaling strategy, achieving a good balance between input resolution, network width, and depth. It performs well on multiple detection tasks, significantly reducing model parameters and computational cost while maintaining high accuracy.

[0067] Continue to refer to Figure 2 The system 200 also includes a fusion subsystem 230. The fusion subsystem 230 can determine a target detection result 206 regarding whether the target image 201 has been modified, based on a first detection result 221 and at least one second detection result 222.

[0068] To improve robustness and detection accuracy, multi-dimensional features are extracted in the preprocessing subsystem 210. The modified detection subsystem 220 uses multiple detection models for learning, resulting in multiple detection results. Therefore, the fusion subsystem 230 can determine the target detection result 206 by integrating multi-dimensional feature information and combining the detection results of multiple models. Specifically, the first detection result 221 and at least one second detection result 222 are combined into a set of detection results. Then, using an attention mechanism, the target score 206 is determined based on this set of detection results.

[0069] As an example, detection results based on different feature dimensions or different detection models can be combined into a set of detection results as input to the fusion subsystem 230. (Reference) Figure 3C Describe an example. Figure 3C A schematic diagram of the architecture 330 of a fusion subsystem 230 according to some embodiments of the present disclosure is shown. Figure 3C As shown, in some embodiments, the detection result 331 may include scores 331-1, 331-2, ..., 331-K, output by different detection models, indicating whether the target image has been modified. These scores may be collectively referred to as scores 331 or individually, where K is a positive integer. Correspondingly, a set of detection results may include a score array 332, or may be considered as a vector of size 1×K.

[0070] The score array 332 is input to the attention module 333. The attention module 333 uses an attention mechanism to determine the corresponding weights for each score 331 in the score array 332. For example, in the attention module 333, the 1×K vector representing the score array 332 is first input to a fully connected layer (K×128), mapping the dimension from 1×K to 1×128. Then, it passes through another fully connected layer (128×K), mapping the dimension back to 1×K. Finally, the output is normalized using the sigmoid function to obtain a weight vector (each value between 0 and 1). Note that the number of scores and the specific dimensions of the feature representations described here are merely exemplary and are not intended to be any limitation.

[0071] In some embodiments, the fusion subsystem 230 weights a set of detection results according to the corresponding weights of the set of detection results to generate a set of weighted detection results. For example, the score array 332 (1×K) is multiplied element-wise with the weight vector (1×K) generated by the attention module 333 to obtain the weighted score.

[0072] In some embodiments, the weighted detection results (e.g., weighted scores) are further fed into the fully connected layer 334. In the fully connected layer 334, information is extracted and fused through a multi-layer fully connected network to determine a target score 206 indicating whether the target image has been modified.

[0073] In the embodiments of this disclosure, an attention mechanism is introduced to dynamically allocate weights, fully leveraging the advantages of multiple feature branches and the model. This approach effectively highlights features that contribute significantly to the detection results while suppressing interference from irrelevant or weakly relevant information, thereby improving the accuracy and robustness of image detection.

[0074] The image detection method described above will be explained below with examples. For illustrative purposes only, system 200 is used as a system for image detection, and the original RGB image that needs to be modified (e.g., tampering detection) is used as an example of the target image.

[0075] In some embodiments, the system 200 can be used to detect whether an acquired image has been modified, for example, in applications such as social media and misinformation verification. The system 200 first acquires the target image to be detected and then begins the detection process for that target image.

[0076] In the detection process, system 200 performs DCT on the target image to extract DCT coefficients for multiple frequency channels. Then, by statistically analyzing the coefficient distribution of each channel, a corresponding feature vector is generated. System 200 can further extract other image features, such as RGB features, YCbCr features, and SRM features. Each feature is input into its corresponding model for analysis. DCT features are processed using a one-dimensional convolutional neural network, while other features are input into depthwise separable convolutional networks or efficient convolutional networks for analysis. Then, system 200 obtains detection scores for each feature.

[0077] System 200 can use an attention-based network model to fuse the detection scores corresponding to each feature, thereby determining the overall detection score of the image. Based on the overall detection score, system 200 can output the result of whether the image has been tampered with.

[0078] In summary, according to the various embodiments of this disclosure, by combining frequency domain and spatial domain features and employing multi-model fusion detection, the accuracy, robustness, and applicability of image tampering detection are significantly improved. Furthermore, by extracting key frequency domain channel information and preserving the frequency features of the modified region during frequency domain transformation, the detection performance can be further optimized while enhancing the detection capability for low-quality images.

[0079] Example process

[0080] Figure 4 A flowchart of an image detection process 400 according to some embodiments of the present disclosure is shown. Process 400 can be implemented at the model application system 160. Reference is made below. Figure 1 Describe the process 400.

[0081] In box 410, the model application system 160 determines a first feature of the target image by performing a transformation from the spatial domain to the frequency domain on the target image. The first feature includes at least one feature vector corresponding to at least one frequency channel.

[0082] In box 420, the model application system 160 determines a first detection result regarding whether the target image has been modified based on the first feature.

[0083] In box 430, model application system 160 determines at least one second detection result regarding whether the target image has been modified based on at least one second feature of the target image, wherein the dimension of the at least one second feature is different from the dimension of the first feature.

[0084] In box 440, model application system 160 determines a target detection result regarding whether the target image has been modified, based on a first detection result and at least one second detection result.

[0085] In some embodiments, determining the first feature includes: dividing the target image into multiple image blocks; determining the corresponding frequency domain information of the multiple image blocks by performing a transformation from the spatial domain to the frequency domain on each of the multiple image blocks, wherein the frequency domain information of each image block includes the corresponding transformation coefficients of the image block in multiple frequency channels; and statistically analyzing the corresponding frequency domain information of the multiple image blocks according to the multiple frequency channels to determine the feature vector corresponding to at least one frequency channel.

[0086] In some embodiments, the transformation from the spatial domain to the frequency domain includes a discrete cosine transform, and the transform coefficients include discrete cosine transform coefficients.

[0087] In some embodiments, at least one frequency channel is selected from a plurality of frequency channels.

[0088] In some embodiments, determining the first detection result includes: using a first detection model based on one-dimensional convolution operations, and based on a first feature, determining the first detection result.

[0089] In some embodiments, at least one second feature includes at least one of the following: a feature represented by a plurality of color channels, a feature represented by a luminance channel and at least one chromatic difference channel, or a feature based on image noise analysis.

[0090] In some embodiments, determining at least one second detection result includes: for a given second feature among at least one second feature, determining one or more detection results corresponding to the given second feature based on one or more second detection models; and determining one or more detection results as part of at least one second detection result.

[0091] In some embodiments, one or more second detection models include at least one of the following: a model based on depthwise separable convolution, or a model based on efficient convolution.

[0092] In some embodiments, the target detection result includes a target score indicating whether the target image has been modified, and determining the target detection result includes: combining a first detection result and at least one second detection result into a set of detection results; and using an attention mechanism to determine the target score based on the set of detection results.

[0093] In some embodiments, determining a target score includes: using an attention mechanism to determine corresponding weights for a set of detection results; weighting the set of detection results according to their corresponding weights to generate a set of weighted detection results; and determining a target score indicating whether a target image has been modified based on the set of weighted detection results.

[0094] Example devices and equipment

[0095] Figure 5 A schematic structural block diagram of an apparatus 500 for image detection according to certain embodiments of the present disclosure is shown. The apparatus 500 may be implemented as or included in a model application system 160. The various modules / components in the apparatus 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0096] As shown in the figure, the apparatus 500 includes a feature determination module 510 configured to determine a first feature of a target image by performing a transformation from the spatial domain to the frequency domain on the target image, the first feature including at least one feature vector corresponding to at least one frequency channel; a first detection result determination module 520 configured to determine a first detection result regarding whether the target image has been modified based on the first feature; a second detection result determination module 530 configured to determine at least one second detection result regarding whether the target image has been modified based on at least one second feature of the target image, the dimension of the at least one second feature being different from the dimension of the first feature; and a target detection result determination module 540 configured to determine a target detection result regarding whether the target image has been modified based on the first detection result and at least one second detection result.

[0097] In some embodiments, the feature determination module 510 is further configured to divide the target image into multiple image blocks; determine the corresponding frequency domain information of the multiple image blocks by performing a transformation from the spatial domain to the frequency domain on each of the multiple image blocks, wherein the frequency domain information of each image block includes the corresponding transformation coefficients of the image block in multiple frequency channels; and perform statistics on the corresponding frequency domain information of the multiple image blocks according to multiple frequency channels to determine the feature vector corresponding to at least one frequency channel.

[0098] In some embodiments, the transformation from the spatial domain to the frequency domain includes a discrete cosine transform, and the transform coefficients include discrete cosine transform coefficients.

[0099] In some embodiments, at least one frequency channel is selected from a plurality of frequency channels.

[0100] In some embodiments, the first detection result determination module 520 is further configured to determine a first detection result based on a first feature using a first detection model based on a one-dimensional convolution operation.

[0101] In some embodiments, at least one second feature includes at least one of the following: a feature represented by a plurality of color channels, a feature represented by a luminance channel and at least one chromatic difference channel, or a feature based on image noise analysis.

[0102] In some embodiments, the second detection result determination module 530 is further configured to, for a given second feature among at least one second feature, determine one or more detection results corresponding to the given second feature based on one or more second detection models; and determine one or more detection results as part of at least one second detection result.

[0103] In some embodiments, one or more second detection models include at least one of the following: a model based on depthwise separable convolution, or a model based on efficient convolution.

[0104] In some embodiments, the target detection result includes a target score indicating whether the target image has been modified, and the target detection result determination module 540 is further configured to combine a first detection result and at least one second detection result into a set of detection results; and to determine a target score based on the set of detection results using an attention mechanism.

[0105] In some embodiments, the target detection result determination module 540 is further configured to utilize an attention mechanism to determine the corresponding weights of a set of detection results; to weight the set of detection results according to the corresponding weights of the set of detection results to generate a set of weighted detection results; and to determine a target score indicating whether the target image has been modified based on the set of weighted detection results.

[0106] Figure 6 A block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 6 The electronic device 600 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 6 The electronic device 600 shown can be used to achieve Figure 1 Terminal equipment 110.

[0107] like Figure 6As shown, electronic device 600 is in the form of a general-purpose electronic device. Components of electronic device 600 may include, but are not limited to, one or more processors or processing units 610, memory 620, storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. Processing unit 610 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 620. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 600.

[0108] Electronic device 600 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 620 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 can be a removable or non-removable medium and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within electronic device 600.

[0109] Electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 6 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 620 may include computer program product 625 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0110] The communication unit 640 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 600 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 600 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0111] Input device 650 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 660 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 600 can also communicate with one or more external devices (not shown) via communication unit 640 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 600, or with any device that enables electronic device 600 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0112] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0113] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0114] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0115] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0117] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. An image detection method, comprising: A first feature of the target image is determined by performing a transformation from the spatial domain to the frequency domain on the target image. The first feature includes at least one feature vector corresponding to at least one frequency channel. Based on the first feature, a first detection result is determined regarding whether the target image has been modified; Based on at least one second feature of the target image, at least one second detection result is determined regarding whether the target image has been modified, wherein the dimension of the at least one second feature is different from the dimension of the first feature; as well as Based on the first detection result and the at least one second detection result, a target detection result is determined regarding whether the target image has been modified.

2. The method according to claim 1, wherein determining the first feature comprises: The target image is divided into multiple image blocks; By performing the transformation from the spatial domain to the frequency domain on each of the plurality of image blocks, the corresponding frequency domain information of the plurality of image blocks is determined, and the frequency domain information of each image block includes the corresponding transformation coefficients of the image block in the plurality of frequency channels; as well as The corresponding frequency domain information of the multiple image blocks is statistically analyzed according to the multiple frequency channels to determine the feature vector corresponding to the at least one frequency channel.

3. The method according to claim 2, wherein the transformation from the spatial domain to the frequency domain includes a discrete cosine transform, and the transformation coefficients include discrete cosine transform coefficients.

4. The method of claim 2, wherein the at least one frequency channel is selected from the plurality of frequency channels.

5. The method according to claim 1, wherein determining the first detection result includes: The first detection result is determined based on the first feature using a first detection model based on one-dimensional convolution operations.

6. The method of claim 1, wherein the at least one second feature comprises at least one of the following: Features represented by multiple color channels Features represented by a luminance channel and at least one chromaticity channel, or Features based on image noise analysis.

7. The method of claim 1, wherein determining the at least one second detection result comprises: For a given second feature among the at least one second feature, one or more detection results corresponding to the given second feature are determined based on one or more second detection models; as well as The one or more detection results are identified as part of the at least one second detection result.

8. The method of claim 7, wherein the one or more second detection models comprise at least one of the following: Models based on depthwise separable convolution, or A model based on efficient convolution.

9. The method of claim 1, wherein the target detection result includes a target score indicating whether the target image has been modified, and determining the target detection result includes: The first detection result and at least one second detection result are combined into a set of detection results; as well as The target score is determined based on the set of detection results using an attention mechanism.

10. The method of claim 9, wherein determining the target score comprises: Using the attention mechanism, the corresponding weights of the set of detection results are determined; The set of detection results is weighted according to the corresponding weights to generate a set of weighted detection results; as well as Based on the set of weighted detection results, a target score indicating whether the target image has been modified is determined.

11. An apparatus for image detection, comprising: The feature determination module is configured to determine a first feature of the target image by performing a transformation from the spatial domain to the frequency domain on the target image, the first feature including at least one feature vector corresponding to at least one frequency channel respectively; The first detection result determination module is configured to determine a first detection result regarding whether the target image has been modified based on the first feature; The second detection result determination module is configured to determine at least one second detection result regarding whether the target image has been modified based on at least one second feature of the target image, wherein the dimension of the at least one second feature is different from the dimension of the first feature; as well as The target detection result determination module is configured to determine, based on the first detection result and the at least one second detection result, whether the target image has been modified.

12. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 10 when executed by the at least one processing unit.

13. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 10.

Images