Image tampering positioning method, device and medium based on deep learning

Abstract

The present invention discloses a method, device and medium for locating image tampering based on deep learning, and relates to the field of artificial intelligence. The method comprises: obtaining a target image to be tampered with and located; inputting images of C channels and full-channel images in the target image into C+1 feature extraction layers based on deep learning, respectively, extracting the depth features of each channel image and the full-channel image; inputting the extracted C+1 depth features and the fusion features of the C+1 depth features into C+2 output layers, respectively, and obtaining C+2 preliminary positioning results about the tampered area in the target image; performing weighted fusion on the C+2 preliminary positioning results according to the sum of squares of deviations from the mean of all pixels in each channel image, and obtaining the final positioning result. The present invention improves the accuracy of locating the tampered area.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence, and in particular to a method, device and medium for locating image tampering based on deep learning. Background Art

[0002] Image tampering location technology aims to detect and locate tampered areas in images to ensure the authenticity and integrity of the image. The harmfulness of image tampering cannot be ignored. With the popularization of image editing technology and software, the impact of tampered images in various fields is becoming increasingly serious.

[0003] Traditional image tampering detection methods mainly rely on artificially designed feature extraction and image statistical analysis. These methods have limited effectiveness when faced with complex tampering techniques and high-quality tampered images, and it is difficult to provide accurate tampering localization. Summary of the invention

[0004] The embodiments of the present invention provide a deep learning-based image tampering location method, device and medium to solve the above-mentioned technical problems.

[0005] In a first aspect, an embodiment of the present invention provides an image tampering location method based on deep learning, characterized in that it includes:

[0006] Acquire a target image to be tampered with;

[0007] Input the images of C channels and the full-channel image in the target image into C+1 feature extraction layers based on deep learning, respectively, to extract the deep features of each channel image and the full-channel image, respectively, where C represents the number of channels of the target image;

[0008] The extracted C+1 depth features and the fusion features of the C+1 depth features are respectively input into C+2 output layers to obtain C+2 preliminary positioning results of the tampered area in the target image;

[0009] According to the sum of squares of the mean deviations of all pixels in each channel image, the C+2 preliminary positioning results are weightedly fused to obtain the final positioning result;

[0010] Among them, each positioning result is an image mask that indicates the tampered area as 1; the sum of squares of each deviation from the mean is used to characterize the pixel consistency of each channel image. The larger the sum of squares of the deviation from the mean, the higher the ability of the corresponding channel to locate the tampered area.

[0011] In a second aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0012] one or more processors;

[0013] a memory for storing one or more programs,

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the image tampering localization method based on deep learning described in any embodiment.

[0015] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image tampering location method based on deep learning described in any embodiment.

[0016] The embodiment of the present invention provides an image tampering positioning method based on deep learning, which uses multiple channels of an image to respectively locate the tampered area and then fuses them, taking into account the characteristics of different channels and extracting more detailed tampering features; at the same time, the possibility of the channel identifying the tampered area is judged by the consistency of pixels of each channel, and the weight parameter is amplified for the channel that may extract more tampering features, and the weight parameter is reduced for the channel with fewer tampering features, thereby realizing differentiated processing of each channel and accurately capturing effective information; finally, the positioning results of each channel are fused at multiple levels to further improve the accuracy of the detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a structural schematic diagram of an image tampering localization model provided by an embodiment of the present invention;

[0019] Figure 2 is a structural schematic diagram of another image tampering localization model provided by an embodiment of the present invention;

[0020] Figure 3 is a flow chart of an image tampering location method based on deep learning provided by an embodiment of the present invention;

[0021] Figure 4 An original image of a color ultrasound image of a new energy battery fluid provided by an embodiment of the present invention;

[0022] Figure 5 By copying and pasting Figure 4 The original image shown is the image after being tampered with;

[0023] Figure 6 yes Figure 5 The R channel image in the image shown;

[0024] Figure 7 yes Figure 5 The G channel image in the image shown;

[0025] Figure 8 yes Figure 5 The B channel image in the image shown;

[0026] Fig. 9 An image tampered with by copying and pasting provided in an embodiment of the present invention;

[0027] Fig.10 for Fig. 9 The true mask of the tampered area in the image shown;

[0028] Fig.11 To adopt the method of the embodiment of the present invention Fig. 9 After the image shown is processed, the mask of the tampered area is located;

[0029] Fig.12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0031] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0032] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0033] The image tampering methods mainly include copying and pasting, that is, copying part of the area in other images and pasting it into the tampered image. Under this tampering method, the tampered area is mostly taken from other images. Therefore, the potential features of the area are inconsistent with the original image, and this inconsistency usually exists in the three channels of RGB at the same time, but the degree of inconsistency in each channel may be different. According to this feature, an image tampering positioning method based on deep learning in an embodiment of the present invention locates the tampered area by extracting the features of the three channels of R, G, and B respectively, and then fuses the positioning results to achieve the purpose of accurately detecting the tampered area, which is particularly suitable for images tampered by copying and pasting.

[0034] In order to illustrate the method, the image tampering model used in this embodiment is first introduced. Figure 1 is a structural diagram of an image tampering localization model provided by an embodiment of the present invention, such as Figure 1 As shown, the model includes 4 branches, each of which includes a feature extraction layer and an output layer. Among them, 3 branches are respectively used to input the R channel image, G channel image and B channel image in the target image to be tampered and located, and the feature extraction layers in the 3 branches are respectively used to extract the depth features of the R channel image, the G channel image and the B channel image, and the output layers of the 3 branches are respectively used to locate the tampered area of ​​the target image according to the depth features of the R channel image, the G channel image and the B channel image. Specifically, the positioning result can be an image mask with the same size as the target image and a one-to-one correspondence between pixel positions, wherein the pixel value in the tampered area is 1 and the pixel value in other areas is 0. The fourth branch is used to input the full channel image of the target image (all 3 channels are included), and the feature extraction layer in the branch is used to extract the depth features of the full channel image, and the output layer is used to locate the tampered area of ​​the target image according to the depth features of the full channel image. Furthermore, the depth features extracted by each feature extraction layer can make spatial features that can reflect tampering traces, including structural features, texture features, noise features, etc.

[0035] In addition, the model also includes a feature fusion layer, which is used to fuse the deep features of the three-channel image and the full-channel image to obtain a fused feature; the fused feature will be input into the fifth output layer, which is used to locate the tampered area in the target image according to the fused feature. Finally, the five positioning results will be weighted and fused into the final positioning result. For the convenience of distinction and description, the five positioning results are all referred to as preliminary positioning results in this embodiment.

[0036] In a specific embodiment, the structures of each feature extraction layer, output layer and feature fusion layer are as follows: Figure 2 As shown. Among them, each feature extraction layer adopts the backbone structure of CNN (Convolutional Neural Networks), and each feature extraction layer includes multiple layers of CNN units. Each layer of CNN units includes a convolution layer and a pooling layer. Each convolution layer uses a multi-channel convolution kernel of the same size. The multi-channel convolution kernel is used to extract features from the input data of the current convolution layer. The extracted deep features are added to the input data and then input into the pooling layer in the same CNN unit. The pooling layer performs a pooling operation on the input features, with the purpose of reducing the size of the deep features while keeping the basic structure of the features unchanged. The reduced-size deep features continue to be input into the multi-channel convolution kernel of the next CNN unit, and this cycle is repeated to gradually increase the receptive field of the deep features until the operations of all CNN units are completed. For example, as Figure 2 As shown, each feature extraction layer includes 5 layers of CNN units, and each layer of CNN units includes a 128-channel 3×3 convolution kernel (i.e., convolution layer) and a pooling layer.

[0037] Each output layer includes another convolution layer, a deconvolution layer and another convolution layer. For the convenience of distinction and description, in this embodiment, each convolution layer in the feature extraction layer is referred to as the first convolution layer, the other convolution layer in each output layer is referred to as the second convolution layer, and the another convolution layer in each output layer is referred to as the third convolution layer. Among them, the second convolution layer adopts a multi-channel convolution kernel with the same number of channels as the first convolution layer, and the convolution kernel is used to perform channel transformation on the deep features output by the feature extraction layer; the deconvolution layer and the convolution layer are inverse operations, and the number of channels is the same as the number of channels in the second convolution layer, which is used to gradually expand the deep features after channel transformation to the same size as the original image; the third convolution layer adopts a single-channel convolution kernel, which is used to transform the features after size expansion into a single channel, that is, the number of channels required for the preliminary positioning result. Among them, the previous layer of the second convolution layer is a pooling layer, which reduces the image; the next layer is a deconvolution layer, which enlarges the image. Adding a second convolutional layer between shrinking and enlarging can prevent the image from being directly enlarged after being shrunk, which would make the last pooling layer meaningless. After the last pooling layer shrinks the image, it is equivalent to increasing the receptive field. The second convolutional layer can extract features again on a larger receptive field. For example, Figure 5 As shown in the figure, the second convolution layer uses a 3×3 convolution kernel with 128 channels, and the third convolution layer uses a 1×1 convolution kernel with 1 channel.

[0038] The feature fusion layer includes a matrix splicing layer and another convolution layer, hereinafter referred to as the fourth convolution layer. The matrix splicing layer is used to splice the depth features corresponding to each channel image and the full channel image (the depth features output by each feature extraction layer) along the channel dimension, and the fourth convolution layer is used to perform a convolution operation on the spliced ​​depth features to obtain fusion features. For example, Figure 5 As shown, the fourth convolutional layer uses a 1×1 convolution kernel with 512 channels.

[0039] It should be noted that the specific structures and parameters of each feature extraction layer, output layer and fusion layer can be flexibly set as needed. Only some exemplary embodiments are given above. The transformation of the structure and parameters within the layer by those skilled in the art on the basis of satisfying the basic functions of each layer falls within the protection scope of the present invention.

[0040] Based on the above model structure, the model training process may optionally include the following steps:

[0041] Step 1: Build a sample image library and mark the tampered area of ​​each sample image. Optionally, the sample image library may include multiple images tampered by copying and pasting, or multiple untampered images. The tampered area in each tampered image will be marked in the form of an image mask, and all elements in each untampered image will be marked as 0.

[0042] Step 2: Use each sample image to train the above-mentioned image tampering positioning model. During the training process, the model parameters are updated by minimizing the difference between the true mask of the tampered area in the sample image and each preliminary positioning result, and the greater the weight of each preliminary positioning result, the greater the impact of the difference between the true mask and each preliminary positioning result on the parameter update.

[0043] Optionally, update the model parameters by minimizing the following loss function:

[0044] Loss function = β 1 ×H(R 1 ,R gt )+β 2 ×H(R 2 ,R gt )+β 3 ×H(R 3 ,R gt )+β 4 ×H(R 4 ,R gt )+β 5 ×H(R 5 ,R gt ) (1)

[0045] Among them, Lossfunction represents the loss function value, R gt represents the true mask, R 1 , R 2 and R 3 They represent the preliminary positioning results corresponding to the R channel image, the G channel image and the B channel image of the sample image, respectively. 4 represents the preliminary positioning result corresponding to the full channel image of the sample image, R 5 represents the preliminary positioning result corresponding to the fusion feature of the sample image (the feature after the deep feature fusion of the R channel image, the G channel image, the B channel image and the full channel image); H(R 1 ,R gt )、H(R 2 ,R gt )、H(R 3 ,R gt )、H(R 4 ,R gt ) and H(R 5 ,R gt ) represent R 1 , R 2 , R 3 , R 4 , R 5 R gt The cross entropy result, β1 , β 2 , β 3 , β 4 and β 5 Respectively represent the weight of each cross entropy result.

[0046] Furthermore,

[0047]

[0048] β 4 =1(5)

[0049] β 5 =β 1 +β 2 +β 3 +β 4 (6)

[0050] Among them, sum(std(Image R ))、sum(std(Image G )) and sum(std(Image B )) represent the mean square sum of the R channel image, G channel image and B channel image respectively. Taking the R channel image as an example, the mean square sum of the image is defined as:

[0051]

[0052] Among them, x i represents the i-th pixel in the R channel image, n represents the number of pixels in the R channel image, and μ represents the average value of all pixels in the R channel image.

[0053] After the model training is completed, Figure 3 1 is a flowchart of a method for locating image tampering based on deep learning provided by an embodiment of the present invention. The method is applicable to locating image areas tampered by copying and pasting, and is executed by an electronic device. Figure 3 As shown, the method specifically includes:

[0054] S110: Acquire a target image to be located for tampering.

[0055] S120. Input the images of C channels and the full-channel image in the target image into C+1 feature extraction layers in the trained image tampering localization model, respectively, and extract the deep features of each channel image and the full-channel image, respectively, where C represents the number of channels of the target image.

[0056] S130, inputting the extracted C+1 depth features and the fusion features of the C+1 depth features into C+2 output layers respectively, and obtaining C+2 preliminary positioning results about the tampered area in the target image respectively.

[0057] S140, performing weighted fusion on the C+2 preliminary positioning results according to the sum of squares of deviations from the mean of all pixels in each channel image to obtain a final positioning result.

[0058] Among them, each positioning result is an image mask that indicates the tampered area as 1; the sum of squares of each deviation from the mean is used to characterize the pixel consistency of each channel image. The larger the sum of squares of the deviation from the mean, the higher the possibility of accurately locating the tampered area on the corresponding channel (that is, the positioning ability).

[0059] Optionally, the C channels are three channels of R, G, and B respectively; and performing weighted fusion on the C+2 preliminary positioning results according to the sum of squares of deviations from the mean of all pixels in the images of each channel to obtain the final positioning result may include the following steps:

[0060] Step 1: Calculate the mean square sum of all pixels in the three channel images respectively. That is, use formula (7) to calculate the mean square sum of all pixels in the three channel images respectively.

[0061] Step 2: The ratio of the mean square sum of the mean differences of each channel image to the sum of the mean square sums of the mean differences of the three channel images is used as the weight of the preliminary positioning result corresponding to each channel. That is, the weight of the preliminary positioning result of each channel is calculated using formula (2)(3)(4).

[0062] Step 3: The sum of the weights of the preliminary positioning results corresponding to the three channels is used as the weight of the preliminary positioning result corresponding to the full-channel image. Corresponding to formula (5), the weight is 1.

[0063] Step 4: The sum of the weights of the preliminary positioning results corresponding to the three channels and the weight of the preliminary positioning results corresponding to the full-channel image is used as the weight of the preliminary positioning result corresponding to the fusion feature. Corresponding to formula (6), the weight is 2.

[0064] Step 5: According to the weights of the preliminary positioning results, the C+2 preliminary positioning results are weighted and fused. Optionally, in order to ensure that the pixel value in the final positioning result does not exceed 1, the 5 weights are first normalized:

[0065]

[0066] Then, according to the normalized weights, the preliminary positioning results are weighted fused:

[0067] R=α1 ×R 1 +α 2 ×R 2 +α 3 ×R 3 +α 4 ×R 4 +α 5 ×R 5 (13)

[0068] Among them, R represents the result after weighted fusion. The pixel values ​​in R that are greater than the set threshold are set to 1, and the remaining pixel values ​​are set to 0, and the final positioning result can be obtained.

[0069] It can be seen that in both the model training stage and the model use stage, the weights of the preliminary positioning results corresponding to each channel image, the full channel image and the fusion feature are calculated according to the sum of squares of the mean deviations of each channel image; the difference between the two stages is that the weight is used to weighted sum the loss function in the model training stage, while the weight is used to weighted sum the positioning results in the use stage. The essence of this is that: if the sum of squares of the mean deviations of a channel in the image is small, it proves that the values ​​of all pixels in the channel are relatively uniform, and the tampered area and the original image area are relatively consistent in color in this channel, then the possibility of identifying tampering traces is small (for example, no tampering traces may be identified in a piece of white paper); conversely, if the sum of squares of the mean deviations of a channel in the image is large, it proves that the values ​​of all pixels in the channel are consistent, then the possibility of identifying (i.e. locating) the tampered area is greater.

[0070] In summary, this embodiment provides an image tampering positioning method based on deep learning, which uses multiple channels of the image to identify the tampered area separately and then fuse them, taking into account the characteristics of different channels, and extracting more detailed tampering features; at the same time, the possibility of the channel identifying the tampered area is judged by the consistency of the pixels of each channel. For channels that may extract more tampering features, the weight parameters are amplified, and for channels with fewer tampering features, the weight parameters are reduced, thereby realizing differentiated processing of each channel and accurately capturing effective information; finally, the positioning results of each channel are fused at multiple levels to further improve the accuracy of the positioning results.

[0071] The effectiveness of the method of this embodiment is demonstrated by two application examples below.

[0072] Application Example 1

[0073] Figure 4This is an original image (i.e., an untampered image) of a color ultrasound image of a new energy battery fluid provided by an embodiment of the present invention. The two dotted lines in the image are coordinate alignment lines in the image display software and can be ignored. The pixel values ​​in the image represent the distribution of the battery fluid in the battery. Most of the pixels in the original image are displayed in blue, indicating that the battery fluid fills the entire battery. Figure 5 The image is tampered with by copying and pasting the original image. The damaged area is added to the original image. The tampered area will be judged as an area with low battery liquid content in the battery liquid ultrasonic detection, causing a large error in the battery liquid content detection. Figure 5 After processing the image in the battery, the tampered area can be located to avoid large errors in battery liquid content detection.

[0074] Specifically, the images of the R channel, G channel, and B channel in the tampered image are as follows: Figure 6 , Figure 7 and Figure 8 As shown (pixel values ​​are between 0 and 255), the sum of squares of the mean deviations of the images of each channel are sum(std(Image R ))=596、sum(std(Image G ))=7198、sum(std(Image B ))=815, the weight corresponding to the 5 preliminary positioning results is β 1 =0.0690, β 2 =0.8362, β 3 =0.0948, β 4 =1,β 5 =2. It can be seen that due to the G channel image ( Figure 7 ) is significantly different from the background area (the area other than the tampered area), so obvious tampering traces can be observed in the G channel; while the tampered parts of the R and B channels are slightly different from the background area, and the tampering traces are very weak. This is due to the single color of the original image. Since the Loss function in the present invention adds equalization parameters of different channels, the influence of the R and B channels can be weakened, and the accuracy of positioning can be improved.

[0075] Application Example 2

[0076] Fig. 9 The present invention provides an image tampered by copying and pasting, which adds an animal image to a landscape image by copying and pasting. The real mask of the tampered area is as follows: Fig.10 The image tampering location method provided in this embodiment is used to Fig. 9After processing the image shown in the figure, the final tampered area mask is as follows Fig.11 As shown, it can be seen that the method of this embodiment has sufficient positioning accuracy.

[0077] Fig.12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Fig.12 As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more. Fig.12 A processor 60 is taken as an example; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected by a bus or other means. Fig.12 The example of connecting through bus is taken in the following.

[0078] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the image tampering positioning method based on deep learning in the embodiment of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 61, that is, realizing the above-mentioned image tampering positioning method based on deep learning.

[0079] The memory 61 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 61 may further include a memory remotely arranged relative to the processor 60, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0080] The input device 62 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 63 may include a display device such as a display screen.

[0081] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image tampering location method based on deep learning of any embodiment.

[0082] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0083] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0084] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0085] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A method for locating image tampering based on deep learning, characterized in that: include: Acquire a target image to be tampered with; Input the images of C channels and the full-channel image in the target image into C+1 feature extraction layers based on deep learning, respectively, to extract the deep features of each channel image and the full-channel image, respectively, where C represents the number of channels of the target image; The extracted C+1 depth features and the fusion features of the C+1 depth features are respectively input into C+2 output layers to obtain C+2 preliminary positioning results of the tampered area in the target image; According to the sum of squares of the mean deviations of all pixels in each channel image, the C+2 preliminary positioning results are weightedly fused to obtain the final positioning result; Among them, each positioning result is an image mask that indicates the tampered area as 1; the sum of squares of each deviation from the mean is used to characterize the pixel consistency of each channel image. The larger the sum of squares of the deviation from the mean, the higher the ability of the corresponding channel to locate the tampered area; The C channels are three channels of R, G and B respectively; The C+2 preliminary positioning results are weightedly fused according to the mean square sum of all pixels in each channel image to obtain the final positioning result, including: Calculate the mean square sum of all pixels in the three channel images respectively; The ratio of the sum of squares of the mean deviations of each channel image to the sum of the sums of squares of the mean deviations of the three channel images is used as the weight of the preliminary positioning result corresponding to each channel; The sum of the weights of the preliminary positioning results corresponding to the three channels is used as the weight of the preliminary positioning result corresponding to the full-channel image; The weights of the preliminary positioning results corresponding to the three channels and the sum of the weights of the preliminary positioning results corresponding to the full-channel image are used as the weight of the preliminary positioning result corresponding to the fusion feature; According to the weight of each preliminary positioning result, the C+2 preliminary positioning results are weightedly fused to obtain the final positioning result.

2. The method according to claim 1, characterized in that: Before inputting the images of C channels and the full-channel image in the target image into C+1 feature extraction layers based on deep learning respectively, the method further includes: Using sample images to train the image tampering localization model composed of the C+1 feature extraction layers and the C+2 output layers; During the training process, the model parameters are updated by minimizing the difference between the true mask of the tampered area in the sample image and each preliminary positioning result, and the greater the weight of each preliminary positioning result, the greater the impact of the difference between the true mask and each preliminary positioning result on the parameter update.

3. The method according to claim 2, characterized in that The updating of the model parameters by minimizing the difference between the real mask of the tampered area in the sample image and each preliminary positioning result includes: Update the model parameters by minimizing the following loss function: Lossfunction=β1×H(R1,R gt )+β2×H(R2,R gt )+β3×H(R3,R gt )+β4×H(R4,R gt )+β5×H(R5,R gt ) Among them, Lossfunction represents the loss function value, R gt represents the true mask, R1, R2 and R3 represent the preliminary positioning results corresponding to the R channel image, G channel image and B channel image of the sample image respectively, R4 represents the preliminary positioning result corresponding to the full channel image of the sample image, and R5 represents the preliminary positioning result corresponding to the fusion feature of the sample image; H(R1,R gt )、H(R2,R gt )、H(R3,R gt )、H(R4,R gt ) and H(R5,R gt ) respectively represent R1, R2, R3, R4, R5 and R gt The cross entropy result of , β1, β2, β3, β4 and β5 represent the weights of each cross entropy result, where, β4＝1 β5＝β1+β2+β3+β4 Among them, sum(std(Image R ))、sum(std(Image G )) and sum(std(Image B )) represent the sum of squared deviations from the mean of the R channel image, the G channel image, and the B channel image respectively.

4. The method according to claim 1, characterized in that The weighted fusion of the C+2 preliminary positioning results is performed according to the weights of the preliminary positioning results to obtain the final positioning result, including: The preliminary positioning results are weighted and fused according to the following formula: R=α1×R1+α2×R2+α3×R3+α4×R4+α5×R5 Among them, R represents the result after weighted fusion, The pixel values ​​in R that are greater than the set threshold are set to 1, and the remaining pixel values ​​are set to 0 to obtain the final positioning result.

5. The method according to claim 1, characterized in that Each feature extraction layer includes multiple layers of CNN units, and each layer of CNN units includes the first convolutional layer and the pooling layer; Among them, each first convolution layer uses a multi-channel convolution kernel with residual connection to extract the deep features of the input data. The extracted deep features are added to the input data and then input into the pooling layer of the same CNN unit. Each pooling layer is used to reduce the size of the deep features while maintaining the feature structure; The reduced depth features continue to be input into the first convolutional layer of the next layer of CNN units, and the cycle continues, gradually increasing the receptive field of the depth features until the operations of each layer of CNN units are completed.

6. The method according to claim 1, characterized in that Each output layer includes a second convolutional layer, a multi-channel deconvolutional layer, and a third convolutional layer; Among them, each second convolutional layer uses the same number of convolution kernels as the first convolutional layer channels to transform the deep features output by the last pooling layer. Each multi-channel deconvolution layer is used to gradually expand the changed features to the same size as the image input to each feature extraction layer; Each single-channel convolution kernel is used to transform the enlarged features into the single channel required for each preliminary positioning result.

7. The method according to claim 1, characterized in that Before respectively inputting the extracted C+1 depth features and the fusion features of the C+1 depth features into C+2 output layers to respectively obtain C+2 preliminary positioning results of the tampered area in the target image, the method further includes: Perform matrix concatenation on the C+1 depth features; The fourth convolutional layer is used to perform a convolution operation on the concatenated matrix to obtain a fusion feature of the C+1 deep features.

8. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the deep learning-based image tampering localization method described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, which, when executed by a processor, implements the image tampering location method based on deep learning as described in any one of claims 1-7.

Images