Fraud image detection method based on CTFNet

Abstract

The invention relates to a fraud image detection method based on a CTFNet, and belongs to the technical field of fraud detection, and the method comprises the following steps: S1, constructing a fraud image training set and an image similarity test set; s2, constructing a CTFNet feature extraction network, and respectively extracting local features, global features and frequency domain high and low frequency features of two data set airspaces; s3, the cosine similarity of the features of the two data sets is calculated, and a similarity threshold value of the fraud image is obtained through adaptive weighted fusion; s4, inputting a to-be-detected image into the CTFNet, and extracting local features, global features and frequency domain high and low frequency features of a spatial domain of the to-be-detected image; s5, calculating the cosine similarity of the features of the to-be-detected image and the fraud image training set, and obtaining the optimal similarity score of the to-be-detected image through adaptive weighted fusion; and S6, comparing the optimal similarity score of the to-be-detected image with a similarity threshold value of the fraud image to obtain a detection result of the to-be-detected image.

Description

Technical Field

[0001] The present invention belongs to the technical field of fraudulent image detection and relates to a fraudulent image detection method based on CTFNet. Background Art

[0002] With the rapid development of the Internet and mass media, data in various forms such as text and images has exploded, which has brought great challenges to the review of related content. Today's anti-fraud technologies mostly use machine learning methods to detect fraudulent websites and fraudulent apps' URLs and HTML, and there are few studies on fraudulent image detection. Fraudulent image detection is committed to detecting and identifying fraudulent behavior in image content, improving the accuracy of the network supervision system's review of fraudulent images, and protecting the property and life safety of netizens.

[0003] Fraudulent image detection includes traditional image forensics methods, fraudulent image detection methods based on machine learning, and fraudulent image detection methods based on deep feature extraction. Traditional image forensics methods focus on analyzing the physical properties, format, and metadata of images to detect signs of tampering. Since normal images on the Internet also contain traces of tampering, it is difficult to accurately detect fraudulent images. Fraudulent image detection methods based on machine learning use machine learning to detect tampering or forgery in images, but the detection accuracy of fraudulent images is low. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a fraud image detection method based on CTFNet. CTFNet consists of CNN, Transformer and frequency domain fusion network. CNN and Transformer extract local features and global features of the image space domain respectively. The frequency domain fusion network extracts high-frequency components of local features and low-frequency components of global features through discrete Fourier transform and filter. By fusing local features, global features and high and low frequency features of the frequency domain, and adaptively assigning different weights, it helps to improve the accuracy of fraud image detection.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A fraud image detection method based on CTFNet includes the following steps:

[0007] S1: Obtain fraudulent images and normal images, and construct fraudulent image training sets and image similarity test sets;

[0008] S2: construct a CTFNet feature extraction network, input the fraudulent image training set and the image similarity test set into CTFNet, and extract the local features, global features, and high and low frequency features in the spatial domain;

[0009] S3: Calculate the cosine similarity of the features of the fraudulent image training set and the image similarity test set respectively, and obtain the similarity threshold of the fraudulent image through adaptive weighted fusion;

[0010] S4: Input the image to be detected into CTFNet to extract the local features, global features, and high and low frequency features of the spatial domain of the image to be detected;

[0011] S5: Calculate the cosine similarity of the features of the image to be detected and the fraudulent image training set respectively, and obtain the best similarity score of the image to be detected through adaptive weighted fusion;

[0012] S6: Compare the best similarity score of the image to be detected with the similarity threshold of the fraudulent image to obtain a detection result of the image to be detected.

[0013] Further, step S1 specifically includes the following steps:

[0014] Collect fraudulent images with obvious fraud characteristics from fraud-related carriers, and manually screen the selected images to construct a fraudulent image training set and an image similarity test set;

[0015] The image enhancement technology based on random strategies and amplitudes is used to expand the fraudulent image training set and image similarity test set: first, a set of image enhancement strategies is defined, including rotation, translation, scaling, flipping, brightness adjustment, contrast adjustment, noise addition, and random erasure. Each time, an image enhancement strategy is randomly selected and its enhancement amplitude is randomly determined. Finally, the fraudulent image training set and image similarity test set are enhanced.

[0016] Normal images are randomly selected from the public dataset and added to the image similarity test set to construct a new image similarity test set.

[0017] Furthermore, in step S2, the CTFNet feature extraction network is composed of a CNN network, a Transformer network, and a frequency domain fusion network; the CNN network is used to learn local information of the image space from images of different scales; the Transformer network is used to capture the global dependency between various positions in the image space; and the frequency domain fusion network is used to extract high and low frequency components of global features and local features.

[0018] Furthermore, the CNN network is specifically as follows: the first 13 convolutional layers of VGG16 are used for local feature extraction. The input image is convolved twice by the convolutional layer with 64 convolutional kernels for the first time, and then maxpooling is used. The second time, the input image is convolved twice by the convolutional layer with 128 convolutional kernels, and then maxpooling is used again. The third time, the input image is convolved twice by three convolutional layers with 512 convolutional kernels, and then maxpooling is used again. The local feature X is output.CNN .

[0019] Furthermore, the Transformer network uses the MHSA (Multi-Head Self-Attention) structure to replace the convolutional layer to extract the global feature X of the image. Transformer The input image is first embedded into a fixed-dimensional vector representation x, and x is linearly transformed to generate three matrices: query Q, key K, and value V. The linear transformation formula is as follows:

[0020] Q=W Q x,K=W K x,V=W V x

[0021] Among them, W Q , W K , W V It is a learnable weight matrix. For each query vector Q at each position, the dot product between it and all key vectors K is calculated to obtain the attention score. The calculation formula is:

[0022]

[0023] Among them, d k is the dimension of the key, used to scale the dot product, and softmax is used to convert the dot product into a probability distribution;

[0024] MHSA is calculated in parallel through multiple sets of different Q, K, and V transformations. Each head focuses on different feature patterns, so that the model captures different global dependencies in different subspaces. The output of MHSA is expressed by the following formula:

[0025] X Transformer =Concat(Attention1,…,Attentionh)W O

[0026] Among them, each head independently calculates self-attention, W O is the output weight matrix for the linear transformation.

[0027] Furthermore, the frequency domain fusion network uses a feature normalization layer to normalize the global features, eliminating the scale difference between global features and local features, so that the dimension of the global features becomes C×H×W;

[0028] The feature normalization layer first uses a fully connected layer to convert the embedding dimension E of the Transformer output to C, and then reshapes the Transformer output to C×H×W by downsampling K=H×W. The class token is discarded during feature alignment and fusion.

[0029] The global features and local features are transformed into the frequency domain through discrete Fourier transform. The discrete Fourier transform formula is as follows:

[0030]

[0031] Among them, x c (m,n) is the output feature of Transformer or CNN, X(u,v) is the feature in the corresponding frequency domain, u and v are the coordinates in the frequency domain, representing the horizontal and vertical components of the frequency;

[0032] Use a bandpass filter to extract the high-frequency component H of the CNN feature in the frequency domain CNN , use a low-pass filter to extract the low-frequency component L of the Transformer feature in the frequency domain Transformer , the bandpass filter and lowpass filter formulas are as follows:

[0033]

[0034] Among them, h H is a bandpass filter, D low and D high is the cutoff frequency of the bandpass filter, h L is a low-pass filter, D high1 is the low-pass filter cutoff frequency;

[0035] High frequency component H of CNN features CNN and the low-frequency component L of the Transformer feature Transformer The calculation formula is as follows:

[0036] H CNN (u,v)=h H *X CNN (u,v)

[0037] L Transformer (u,v)=h L *X Transfomer (u,v)

[0038] After applying bandpass and low-pass filters in the frequency domain, the high-frequency features H of CNN are CNN and Transformer’s low-frequency features L Transformer Combined to form fusion feature X Fusion :

[0039] X Fusion =H CNN (u,v)+L Transfomer (u,v)

[0040] The fraudulent image training set and the image similarity test set are input into CTFNet, and the local feature set obtained from the fraudulent image training set is recorded as the local feature F of the fraudulent image training set. 1j , the global feature set is recorded as the global feature F of the fraud image training set 2j , the frequency domain fusion feature set is recorded as the frequency domain fusion feature F of the fraud image training set 3j ; The local feature set obtained from the image similarity test set is recorded as the local feature f of the image similarity test set 1t , the global feature set is recorded as the global feature f of the image similarity test set 2t , the frequency domain fusion feature set is recorded as the frequency domain fusion feature f of the image similarity test set 3t , where j and t represent the number of images in the training set and the test set, respectively.

[0041] Furthermore, in step S3, cosine similarity is used to measure the similarity between two features. The cosine similarity formula is as follows:

[0042]

[0043] Among them, i takes values ​​of 1, 2, and 3, and F ij ·f it Denotes the eigenvector F ij and f it The dot product of ||F ij || and ||f it || represents the eigenvector F ij and f it Length;

[0044] According to the cosine similarity formula, the local features F of the fraud image training set are obtained 1j Similarity test set with local features f 1t feature similarity; each image t in the image similarity test set and each image j in the fraudulent image training set have a local feature similarity, and the maximum value represents the best feature similarity K1 of the local features between the image t in the image similarity test set and the fraudulent image training set; similarly, the best feature similarity K2 of the global features between the image t in the image similarity test set and the fraudulent image training set and the best feature similarity K3 of the frequency domain fusion features are obtained;

[0045] Each image in the image similarity test set has a local feature best feature similarity K1, a global feature best feature similarity K2, and a frequency domain fusion feature best feature similarity K3 with the fraudulent image training set. K1, K2, and K3 are adaptively weighted fused through an adaptive weighted fusion matrix to obtain the best similarity score K between each image in the image similarity test set and the fraudulent image training set. The formula is as follows:

[0046] K=[a,b,c]*[K1,K2,K3] T =aK1+bK2+cK3

[0047] Among them, [a, b, c] is the adaptive weighted fusion matrix, and the weight coefficients a, b, and c are obtained according to K1, K2, and K3. The formula is as follows:

[0048]

[0049] b, b, and c are automatically assigned according to the relative values ​​of K1, K2, and K3, and weights are automatically assigned based on the contribution of different features;

[0050] The similarity threshold of the fraudulent image is set to different values. If K is greater than or equal to the similarity threshold of the fraudulent image, it is a fraudulent image, otherwise it is a normal image. The detection accuracy of the normal image and the fraudulent image in the image similarity test set is calculated respectively, that is, the normal image is detected as a normal image, and the fraudulent image is detected as a fraudulent image. When the sum of the detection accuracy of the normal image and the fraudulent image is the largest, the preset fraudulent image similarity threshold is the fraudulent image similarity threshold K'.

[0051] Furthermore, in step S4, the image to be detected is input into CTFNet, the local features obtained from the image to be detected are recorded as the local features f1 of the image to be detected, the global features are recorded as the global features f2 of the image to be detected, and the frequency domain fusion features are recorded as the frequency domain fusion features f3 of the image to be detected.

[0052] Furthermore, in step S5, the cosine similarity formula is used to calculate the local feature f1 of the image to be detected and the local feature F1 of the fraudulent image training set. 1j The feature similarity of j is calculated. Since j has different values, each image j in the image to be detected and the fraudulent image training set has a local feature similarity. The maximum value represents the best feature similarity of the local features of the image to be detected and the fraudulent image training set, which is recorded as the best similarity score k1 of the local features of the image to be detected. Similarly, the best similarity score k2 of the global features of the image to be detected and the fraudulent image training set and the best similarity score k3 of the frequency domain fusion features are obtained. The best similarity score k between the image to be detected and the fraudulent image training set is obtained by adaptively weighted fusion of k1, k2, and k3.

[0053] Furthermore, in step S6, the best similarity score k between the image to be detected and the fraudulent image training set is compared with the fraudulent image similarity threshold K'. If k≥K', the image to be detected is a fraudulent image, otherwise the image to be detected is a normal image.

[0054] The beneficial effects of the present invention are as follows: CTFNet is composed of CNN, Transformer, and frequency domain fusion network. CNN can learn local information of the image space from images of different scales. Transformer captures the global dependency between various positions in the image space. The frequency domain fusion network extracts high and low frequency components of global and local features to better suppress noise. By fusing local features of the image, global features, and high and low frequency features in the frequency domain, and adaptively assigning different weights, it helps to improve the accuracy of fraud image detection.

[0055] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0057] Figure 1 This is a flow chart of fraud image detection based on CTFNet of the present invention;

[0058] Figure 2 This is the CTFNet network structure diagram of the present invention. DETAILED DESCRIPTION

[0059] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0060] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0061] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0062] like Figure 1 As shown, a fraud image detection method based on CTFNet includes the following steps:

[0063] S1: Obtain fraudulent images and normal images, and construct a fraudulent image training set and an image similarity test set. Optionally, in this embodiment, fraudulent images with obvious fraudulent features are collected from fraudulent carriers such as fraudulent websites and fraudulent apps, and the selected images are manually screened to construct a fraudulent image training set and an image similarity test set. In order to further expand the amount of training data, an image enhancement technology based on random strategies and amplitudes is used to expand the fraudulent image training set and the image similarity test set: first define an image enhancement strategy set, including rotation, translation, scaling, flipping, brightness adjustment, contrast adjustment, adding noise, random erasing, etc., randomly select an image enhancement strategy each time, and randomly determine its enhancement amplitude, and finally enhance the fraudulent image training set and the image similarity test set. In order to improve the accuracy of model training, normal images are randomly extracted from the public data set ImageNet and added to the image similarity test set to construct a new image similarity test set.

[0064] S2: Construct a CTFNet feature extraction network, input the above training set and test set into CTFNet, and extract the local features, global features, and high and low frequency features in the spatial domain. Figure 2 As shown in the figure, CTFNet consists of CNN, Transformer, and frequency domain fusion network. The CNN network learns local information in the image space from images of different scales. The Transformer network captures the global dependencies between various positions in the image space. The frequency domain fusion network extracts high and low frequency components of global features and local features.

[0065] VGG is a type of CNN network. The present invention uses the first 13 convolution layers of VGG16 (the number of channels is 64, 128, and 512, respectively, and is separated by the maximum pooling layer) to extract local features. The input image is convolved twice by the convolution layer with 64 convolution kernels for the first time, and then maxpooling is used. The second time, the input image is convolved twice by the convolution layer with 128 convolution kernels, and then maxpooling is used again. The third time, the input image is convolved twice by three convolution layers with 512 convolution kernels, and then maxpooling is used again. The local feature X is output.CNN .

[0066] The Transformer network uses the MHSA (Multi-Head Self-Attention) structure to replace the convolutional layer to extract the global features of the image X Transformer , the input image is first embedded into a fixed-dimensional vector representation x, and x is linearly transformed to generate three matrices: query (Query, Q), key (Key, K), and value (Value, V). The linear transformation formula is as follows:

[0067] Q=W Q x,K=W K x,V=W V x

[0068] Among them, W Q , W K , W V It is a learnable weight matrix. For each query vector Q at each position, the dot product between it and all key vectors K is calculated to obtain the attention score. The calculation formula is:

[0069]

[0070] Among them, d k is the dimension of the key, used to scale the dot product, and softmax is used to convert the dot product into a probability distribution.

[0071] MHSA is calculated in parallel through multiple sets of different Q, K, V transformations (called multiple "heads"), each head focuses on different feature patterns, so that the model captures different global dependencies in different subspaces. The output of MHSA is expressed by the following formula:

[0072] X Transformer =Concat(Attention1,…,Attentionh)W O

[0073] Among them, each head independently calculates self-attention, W O is the output weight matrix for linear transformation. The multi-head mechanism further enriches the model's ability to understand different features, allowing MHSA to extract complex features with global context.

[0074] The feature dimensions of CNN and Transformer are not consistent. The dimension of CNN feature map is C×H×W, where C represents the number of channels, H represents the height, and W represents the width; while the dimension of Transformer feature map is (K+1)×E, where K represents the number of images, E represents the embedding dimension, and 1 represents the class token, which represents an additional vector. The frequency domain fusion network uses the feature normalization layer to normalize the global features, eliminating the scale difference between the global features and the local features, so that the global feature dimension becomes C×H×W.

[0075] The feature normalization layer first uses a fully connected layer to convert the embedding dimension E of the Transformer output to C, and then reshapes the Transformer output to C×H×W by downsampling K=H×W. The class token is discarded during feature alignment and fusion.

[0076] If global features and local features are directly merged, feature redundancy may occur and effective features of different frequencies cannot be well distinguished. The present invention transforms global features and local features into the frequency domain through discrete Fourier transform. The discrete Fourier transform formula is as follows:

[0077]

[0078] Among them, x c (m,n) is the output feature of Transformer or CNN, X(u,v) is the corresponding feature in the frequency domain, u and v are the coordinates in the frequency domain, representing the horizontal and vertical components of the frequency.

[0079] Since CNN is good at extracting high-frequency information in images and Transformer is good at extracting low-frequency information in images, a bandpass filter is used to extract the high-frequency component H of CNN features in the frequency domain. CNN , use a low-pass filter to extract the low-frequency component L of the Transformer feature in the frequency domain Transformer , the bandpass filter and lowpass filter formulas are as follows:

[0080]

[0081] Among them, h H is a bandpass filter, D low and D high is the cutoff frequency of the bandpass filter, h L is a low-pass filter, D high1 is the low-pass filter cutoff frequency.

[0082] High frequency component H of CNN features CNN and the low-frequency component L of the Transformer feature TransformerThe calculation formula is as follows:

[0083] H CNN (u,v)=h H *X CNN (u,v)

[0084] L Transformer (u,v)=h L *X Transfomer (u,v)

[0085] After applying bandpass and low-pass filters in the frequency domain, the high-frequency features H of CNN are CNN and Transformer’s low-frequency features L Transformer Combined to form fusion feature X Fusion :

[0086] X Fusion =H CNN (u,v)+L Transfomer (u,v)

[0087] CTFNet combines the local feature extraction capability of CNN, the global dependency modeling capability of Transformer, and the noise suppression and frequency component extraction capability of frequency domain fusion network, enhancing the flexibility and robustness of image representation, making it perform well in fraud image detection tasks.

[0088] The fraudulent image training set and the image similarity test set are input into CTFNet, and the local feature set obtained from the fraudulent image training set is recorded as the local feature F of the fraudulent image training set. 1j , the global feature set is recorded as the global feature F of the fraud image training set 2j , the frequency domain fusion feature set is recorded as the frequency domain fusion feature F of the fraud image training set 3j ; The local feature set obtained from the image similarity test set is recorded as the local feature f of the image similarity test set 1t , the global feature set is recorded as the global feature f of the image similarity test set 2t , the frequency domain fusion feature set is recorded as the frequency domain fusion feature f of the image similarity test set 3t , where j and t represent the number of images in the training set and the test set, respectively.

[0089] S3: Calculate the cosine similarity of the features of the fraudulent image training set and the image similarity test set respectively, and obtain the similarity threshold of the fraudulent image through adaptive weighted fusion. Optionally, cosine similarity is used to measure the similarity between two features. The cosine similarity formula is as follows:

[0090]

[0091] Among them, i takes values ​​of 1, 2, and 3, and Fij ·f it Denotes the eigenvector F ij and f it The dot product of ||F ij || and ||f it || represents the eigenvector F ij and f it Length.

[0092] According to the cosine similarity formula, the local features F of the fraud image training set are obtained 1j Similarity test set with local features f 1t Since j and t have different values, each image t in the image similarity test set has a local feature similarity with each image j in the fraudulent image training set. The maximum value represents the best local feature similarity K1 between the image t in the image similarity test set and the fraudulent image training set. Similarly, the best global feature similarity K2 and the best frequency domain fusion feature similarity K3 between the image t in the image similarity test set and the fraudulent image training set can be obtained.

[0093] Each image in the image similarity test set has a local feature best feature similarity K1, a global feature best feature similarity K2, and a frequency domain fusion feature best feature similarity K3 with the fraudulent image training set. K1, K2, and K3 are adaptively weighted fused through the adaptive weighted fusion matrix to obtain the best similarity score K between each image in the image similarity test set and the fraudulent image training set. The formula is as follows:

[0094] K=[a,b,c]*[K1,K2,K3] T =aK1+bK2+cK3

[0095] Among them, [a, b, c] is the adaptive weighted fusion matrix, and the weight coefficients a, b, and c are obtained according to K1, K2, and K3. The formula is as follows:

[0096]

[0097] a, b, and c are automatically assigned according to the relative values ​​of K1, K2, and K3, and weights are automatically assigned based on the contributions of different features, reducing errors caused by human intervention and obtaining a more accurate and representative optimal similarity score K.

[0098] The similarity threshold of the fraudulent image is set to different values ​​(between 0 and 1). If K is greater than or equal to the similarity threshold of the fraudulent image, it is a fraudulent image, otherwise it is a normal image. The detection accuracy of normal images and fraudulent images in the image similarity test set is calculated respectively, that is, normal images are detected as normal images, and fraudulent images are detected as fraudulent images. When the sum of the detection accuracy of normal images and fraudulent images is the largest, the preset fraudulent image similarity threshold is the fraudulent image similarity threshold K'.

[0099] S4: Input the image to be detected into CTFNet, and extract the local features, global features, and high and low frequency features in the spatial domain of the image to be detected. Optionally, input the image to be detected into CTFNet, and the local features obtained from the image to be detected are recorded as the local features f1 of the image to be detected, the global features are recorded as the global features f2 of the image to be detected, and the frequency domain fusion features are recorded as the frequency domain fusion features f3 of the image to be detected.

[0100] S5: Calculate the cosine similarity of the features of the image to be detected and the fraudulent image training set respectively, and obtain the best similarity score of the image to be detected by adaptive weighted fusion. Optionally, the local feature f1 of the image to be detected and the local feature F1 of the fraudulent image training set are obtained by the cosine similarity formula. 1j The feature similarity of j is obtained. Since j has different values, each image j in the training set of the image to be detected and the fraudulent image has a local feature similarity. The maximum value represents the best feature similarity of the local features of the image to be detected and the fraudulent image training set, which is recorded as the best similarity score k1 of the local features of the image to be detected. Similarly, the best similarity score k2 of the global features of the training set of the image to be detected and the fraudulent image and the best similarity score k3 of the frequency domain fusion features can be obtained. The best similarity score k of the training set of the image to be detected and the fraudulent image can be obtained by adaptively weighted fusion of k1, k2, and k3. The formula is the same as that described in S3.

[0101] S6: Compare the best similarity score of the image to be detected with the similarity threshold of the fraudulent image to obtain the detection result of the image to be detected. Optionally, compare the best similarity score k between the image to be detected and the fraudulent image training set with the fraudulent image similarity threshold K'. If k ≥ K', the image to be detected is a fraudulent image, otherwise the image to be detected is a normal image.

[0102] In the above embodiments, the description's reference to "this embodiment" indicates that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily all refer to the same embodiment.

[0103] In the above-described embodiments, although the invention has been described in conjunction with specific embodiments of the invention, many substitutions, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other storage structures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed. Embodiments of the invention are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims.

[0104] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0105] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0106] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0107] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0108] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0109] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0110] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0111] The present invention can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0112] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A fraud image detection method based on CTFNet, characterized by: The following steps are involved: S1: Obtain fraudulent images and normal images, and construct fraudulent image training sets and image similarity test sets; S2: construct a CTFNet feature extraction network, input the fraudulent image training set and the image similarity test set into CTFNet, and extract the local features, global features, and high and low frequency features in the spatial domain; S3: Calculate the cosine similarity of the features of the fraudulent image training set and the image similarity test set respectively, and obtain the similarity threshold of the fraudulent image through adaptive weighted fusion; S4: Input the image to be detected into CTFNet to extract the local features, global features, and high and low frequency features of the spatial domain of the image to be detected; S5: Calculate the cosine similarity of the features of the image to be detected and the fraudulent image training set respectively, and obtain the best similarity score of the image to be detected through adaptive weighted fusion; S6: Compare the best similarity score of the image to be detected with the similarity threshold of the fraudulent image to obtain a detection result of the image to be detected.

2. The fraud image detection method based on CTFNet according to claim 1, characterized in that: Step S1 specifically includes the following steps: Collect fraudulent images with obvious fraud characteristics from fraud-related carriers, and manually screen the selected images to construct a fraudulent image training set and an image similarity test set; The image enhancement technology based on random strategies and amplitudes is used to expand the fraudulent image training set and image similarity test set: first, a set of image enhancement strategies is defined, including rotation, translation, scaling, flipping, brightness adjustment, contrast adjustment, noise addition, and random erasure. Each time, an image enhancement strategy is randomly selected and its enhancement amplitude is randomly determined. Finally, the fraudulent image training set and image similarity test set are enhanced. Normal images are randomly selected from the public dataset and added to the image similarity test set to construct a new image similarity test set.

3. The fraud image detection method based on CTFNet according to claim 1, characterized in that: In step S2, the CTFNet feature extraction network is composed of a CNN network, a Transformer network, and a frequency domain fusion network; the CNN network is used to learn local information of the image space from images of different scales; the Transformer network is used to capture the global dependency between various positions in the image space; and the frequency domain fusion network is used to extract high and low frequency components of global features and local features.

4. The fraud image detection method based on CTFNet according to claim 1, characterized in that: The CNN network is specifically as follows: the first 13 convolutional layers of VGG16 are used for local feature extraction. The input image is convolved twice by the convolutional layer with 64 convolutional kernels for the first time, and then maxpooling is used. The second time, the input image is convolved twice by the convolutional layer with 128 convolutional kernels, and then maxpooling is used again. The third time, the input image is convolved twice by three convolutional layers with 512 convolutional kernels, and then maxpooling is used again. The local feature X is output. CNN .

5. The fraud image detection method based on CTFNet according to claim 1, characterized in that: The Transformer network uses the MHSA (Multi-Head Self-Attention) structure to replace the convolutional layer to extract the global feature X of the image. Transformer The input image is first embedded into a fixed-dimensional vector representation x, and x is linearly transformed to generate three matrices: query Q, key K, and value V. The linear transformation formula is as follows: Q=W Q x,K=W K x,V=W V x Among them, W Q , W K , W V It is a learnable weight matrix. For each query vector Q at each position, the dot product between it and all key vectors K is calculated to obtain the attention score. The calculation formula is: Among them, d k is the dimension of the key, used to scale the dot product, and softmax is used to convert the dot product into a probability distribution; MHSA is calculated in parallel through multiple sets of different Q, K, and V transformations. Each head focuses on different feature patterns, so that the model captures different global dependencies in different subspaces. The output of MHSA is expressed by the following formula: X Transformer =Concat(Attention1,…,Attention h )W O Among them, each head independently calculates self-attention, W O is the output weight matrix for the linear transformation.

6. The fraud image detection method based on CTFNet according to claim 1, characterized in that: The frequency domain fusion network uses the feature normalization layer to normalize the global features, eliminating the scale difference between the global features and the local features, so that the dimension of the global features becomes C×H×W; The feature normalization layer first uses a fully connected layer to convert the embedding dimension E of the Transformer output to C, and then reshapes the Transformer output to C×H×W by downsampling K=H×W. The class token is discarded during feature alignment and fusion. The global features and local features are transformed into the frequency domain through discrete Fourier transform. The discrete Fourier transform formula is as follows: Among them, x c (m,n) is the output feature of Transformer or CNN, X(u,v) is the feature in the corresponding frequency domain, u and v are the coordinates in the frequency domain, representing the horizontal and vertical components of the frequency; Use a bandpass filter to extract the high-frequency component H of the CNN feature in the frequency domain CNN , use a low-pass filter to extract the low-frequency component L of the Transformer feature in the frequency domain Transformer , the bandpass filter and lowpass filter formulas are as follows: Among them, h H is a bandpass filter, D low and D high is the cutoff frequency of the bandpass filter, h L is a low-pass filter, D high1 is the low-pass filter cutoff frequency; High-frequency component H of CNN features CNN and the low-frequency component L of the Transformer feature Transformer The calculation formula is as follows: H CNN (u,v)=h H *X CNN (u,v) L T ransformer(u,v)=h L *X T ransfomer(u,v) After applying bandpass and low-pass filters in the frequency domain, the high-frequency features H of CNN are CNN and Transformer’s low-frequency features L Transformer Combined to form fusion feature X Fusion : X F usion=H CNN (u,v)+L T ransfomer(u,v) The fraudulent image training set and the image similarity test set are input into CTFNet, and the local feature set obtained from the fraudulent image training set is recorded as the local feature F of the fraudulent image training set. 1j , the global feature set is recorded as the global feature F of the fraud image training set 2j , the frequency domain fusion feature set is recorded as the frequency domain fusion feature F of the fraud image training set 3j ; The local feature set obtained from the image similarity test set is recorded as the local feature f of the image similarity test set 1t , the global feature set is recorded as the global feature f of the image similarity test set 2t , the frequency domain fusion feature set is recorded as the frequency domain fusion feature f of the image similarity test set 3t , where j and t represent the number of images in the training set and the test set, respectively.

7. The fraud image detection method based on CTFNet according to claim 1, characterized in that: In step S3, cosine similarity is used to measure the similarity between two features. The cosine similarity formula is as follows: Among them, i takes values ​​of 1, 2, and 3, and F ij ·f it Denotes the eigenvector F ij and f it The dot product of ||F ij || and ||f it || represents the eigenvector F ij and f it Length; According to the cosine similarity formula, the local features F of the fraud image training set are obtained 1j Similarity test set with local features f 1t feature similarity; each image t in the image similarity test set and each image j in the fraudulent image training set have a local feature similarity, and the maximum value represents the best feature similarity K1 of the local features between the image t in the image similarity test set and the fraudulent image training set; similarly, the best feature similarity K2 of the global features between the image t in the image similarity test set and the fraudulent image training set and the best feature similarity K3 of the frequency domain fusion features are obtained; Each image in the image similarity test set has a local feature best feature similarity K1, a global feature best feature similarity K2, and a frequency domain fusion feature best feature similarity K3 with the fraudulent image training set. K1, K2, and K3 are adaptively weighted fused through an adaptive weighted fusion matrix to obtain the best similarity score K between each image in the image similarity test set and the fraudulent image training set. The formula is as follows: K=[a,b,c]*[K1,K2,K3] T =aK1+bK2+cK3 Among them, [a, b, c] is the adaptive weighted fusion matrix, and the weight coefficients a, b, and c are obtained according to K1, K2, and K3. The formula is as follows: a, b, and c are automatically assigned based on the relative values ​​of K1, K2, and K3, and weights are automatically assigned based on the contribution of different features; The similarity threshold of the fraudulent image is set to different values. If K is greater than or equal to the similarity threshold of the fraudulent image, it is a fraudulent image, otherwise it is a normal image. The detection accuracy of the normal image and the fraudulent image in the image similarity test set is calculated respectively, that is, the normal image is detected as a normal image, and the fraudulent image is detected as a fraudulent image. When the sum of the detection accuracy of the normal image and the fraudulent image is the largest, the preset fraudulent image similarity threshold is the fraudulent image similarity threshold K'.

8. The fraud image detection method based on CTFNet according to claim 1, characterized in that: In step S4, the image to be detected is input into CTFNet, the local features obtained from the image to be detected are recorded as the local features f1 of the image to be detected, the global features are recorded as the global features f2 of the image to be detected, and the frequency domain fusion features are recorded as the frequency domain fusion features f3 of the image to be detected.

9. The fraud image detection method based on CTFNet according to claim 1, characterized in that: In step S5, the cosine similarity formula is used to calculate the local feature f1 of the image to be detected and the local feature F1 of the fraudulent image training set. 1j Since j has different values, each image j in the image to be detected and the fraud image training set has a local feature similarity. The maximum value represents the best local feature similarity between the image to be detected and the fraud image training set. Denote it as the best similarity score k1 of the local features of the image to be detected; similarly, the best similarity score k2 of the global features of the image to be detected and the fraudulent image training set and the best similarity score k3 of the frequency domain fusion features are obtained; k1, k2, and k3 are adaptively weighted fused to obtain the best similarity score k between the image to be detected and the fraudulent image training set.

10. The fraud image detection method based on CTFNet according to claim 1, characterized in that: In step S6, the best similarity score k between the image to be detected and the fraudulent image training set is compared with the fraudulent image similarity threshold K'. If k≥K', the image to be detected is a fraudulent image, otherwise the image to be detected is a normal image.

Images