ACDSee image tampering positioning method and device based on large model, terminal and medium

Abstract

The invention provides an ACDSee image tampering positioning method and device based on a large model, a terminal and a medium. The method comprises the following steps: constructing a training data set containing an ACDSee tampering image and a tampering mark image corresponding to the ACDSee tampering image; constructing an image tampering positioning network model based on a preset large model encoder and a preset backbone network; training the image tampering positioning network model by using the training data set; and inputting a to-be-detected image into the trained image tampering positioning network model to identify and position a tampering area of the to-be-detected image, and obtaining a tampering area positioning result. According to the method, the constructed image tampering positioning network model is trained by using the constructed training data set containing the ACDSEE tampered image and the corresponding tampered marked image, so that the network model capable of accurately positioning the tampered area on the ACDSEE tampered image is obtained, and the generalization of the image tampering positioning network model on the ACDSEE tampered image is improved.

Description

Technical Field

[0001] The present invention relates to the field of multimedia information security, and in particular to a large-model-based ACDSee image tampering positioning method, device, terminal and medium. Background Art

[0002] In recent years, with the development of artificial intelligence technology, the threshold for image editing has become lower and lower. For some ulterior motives, some lawless elements will tamper with digital images and profit from them. These tampered images are widely spread through social media, posing great risks to social stability and personal privacy.

[0003] ACDSee is a popular image browsing and editing software. In previous versions, ACDSee was mainly used for browsing and managing images. In recent years, with the continuous improvement of its editing functions and the embedding of AI functions, it is easy for users to use ACDSee to edit and tamper with image content. In addition, ACDSee has a large user base, so there may be a problem that a large number of images tampered with by ACDSee are circulated on the Internet, causing serious impact.

[0004] At present, although people have conducted extensive research in the field of image forensics, most of them are aimed at images tampered with by Photoshop or Deepfake. Due to the use of different algorithms and processing technologies, ACDSee will show different characteristics in image editing. The existing deep learning-based image tampering localization method cannot show satisfactory generalization performance on images tampered with by ACDSee software.

[0005] In summary, the existing deep learning-based image tampering location technology has poor generalization performance for images edited and tampered by ACDSee software. Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve. Summary of the invention

[0006] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a method, device, terminal and medium for locating ACDSee image tampering based on a large model are provided, aiming to solve the problem that the existing image tampering locating technology is not generalizable to images edited and tampered by ACDSee.

[0007] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0008] A large-model-based ACDSee image tampering location method, wherein the method comprises:

[0009] Construct a training dataset containing ACDSee tampered images and their corresponding tampered marked images;

[0010] Construct an image tampering localization network model based on a preset large model encoder and a preset backbone network;

[0011] Using the training data set to train the image tampering localization network model to obtain a trained image tampering localization network model;

[0012] The image to be tested is input into the trained image tampering location network model to identify and locate the tampered area of ​​the image to be tested, and obtain the corresponding tampered area location result.

[0013] In one implementation, constructing a training data set containing ACDSee tampered images and their corresponding tampered marked images includes:

[0014] The tampering tool in the ACDSee software is called by a pre-written image tampering script program, and the acquired original image is tampered with according to the tampering operation, so as to obtain a training data set including the ACDSee tampered image and its corresponding tampered marked image;

[0015] The tampering tool and the tampering operation are the tools used and the operations performed when the actual ACDSee tampering image constructed by the ACDSee software, and the image tampering script program is a script program pre-written based on the tampering tool and the tampering operation.

[0016] In one implementation, tampering the acquired original image includes:

[0017] Smear and tamper with any position on the acquired original image;

[0018] Or, expanding a corresponding blank area in any direction in the acquired original image, and generating a tampered image in the blank area;

[0019] Alternatively, an original foreground image and an original background image of the acquired original image are determined, and any area in the original foreground image is spliced ​​to any position in the original background image.

[0020] In one implementation, the preset backbone network is a network of an encoder and decoder structure that can perform image segmentation tasks, the preset large model encoder is an encoder pre-trained in an image segmentation large model based on the encoder and decoder structure, and the encoder of the preset backbone network for extracting local features is interconnected with the preset large model encoder for extracting global features.

[0021] In one implementation, the preset large model encoder includes a plurality of sequentially connected feature extraction blocks and each of the feature extraction blocks is connected in parallel with a low-rank adapter, and the feature extraction block is a transformer block supporting window attention and residual connection;

[0022] Furthermore, the feature extraction block includes a local attention feature extraction block and a global attention feature extraction block;

[0023] Wherein, the local attention feature extraction block is a feature extraction block using local window attention, and the size of the local window is a preset patch size in the image tampering localization network model;

[0024] The global attention feature extraction block is a feature extraction block using global window attention, and the size of the global window is the image size input to the image tampering localization network model.

[0025] In one implementation, the number of the feature extraction blocks is an integer multiple of three, and the feature extraction blocks are connected in sequence in such a way that three feature extraction blocks form a group, and the three feature extraction blocks are sequentially two local attention feature extraction blocks and one global attention feature extraction block.

[0026] In one implementation, the using the training data set to train the image tampering localization network model to obtain a trained image tampering localization network model includes:

[0027] Freeze the weights corresponding to the feature extraction blocks in the pre-loaded large model encoder;

[0028] Inputting the training data set into the image tampering localization network model for training to update the weights of the non-frozen areas, thereby obtaining a trained image tampering localization network model;

[0029] The weights corresponding to the feature extraction blocks are the weights obtained by training the large image segmentation model on an image segmentation data set, and the non-frozen area includes the encoder and decoder of the backbone network and the low-rank adapter.

[0030] The present invention also discloses a large-model-based image tampering positioning device, wherein the device comprises:

[0031] A data set construction module, used to construct a training data set including ACDSee tampered images and their corresponding tampered marked images;

[0032] A model building module, used to build an image tampering localization network model based on a preset large model encoder and a preset backbone network;

[0033] A model training module, used to train the image tampering localization network model using the training data set to obtain a trained image tampering localization network model;

[0034] The tampered area positioning module is used to input the image to be tested into the trained image tampering positioning network model to identify and locate the tampered area of ​​the image to be tested, and obtain the corresponding tampered area positioning result.

[0035] The present invention also discloses a terminal, which includes: a memory, a processor, and a large model-based image tampering location program stored in the memory and executable on the processor. When the large model-based image tampering location program is executed by the processor, the steps of the large model-based ACDSee image tampering location method are implemented as described above.

[0036] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the ACDSee image tampering positioning method based on a large model as described above.

[0037] The present invention provides an ACDSee image tampering positioning method, device, terminal and medium based on a large model. The ACDSee image tampering positioning method based on a large model includes: constructing a training data set including ACDSee tampered images and their corresponding tampered marked images; constructing an image tampering positioning network model based on a preset large model encoder and a preset backbone network; training the image tampering positioning network model using the training data set to obtain a trained image tampering positioning network model; inputting a test image into the trained image tampering positioning network model to identify and locate the tampered area of ​​the test image, and obtaining a corresponding tampering area positioning result. It can be seen that the present invention uses the ACDSee tampered images and their corresponding tampered marked images in the constructed training data set as training samples, and then uses the training samples to train the image tampering localization network model constructed based on the preset large model encoder and the preset backbone network, so as to obtain a network model that can accurately locate the tampered area on the ACDSee tampered image, that is, the trained image tampering localization network model can achieve accurate positioning of the editing behavior on the ACDSee tampered image, and thus can greatly improve the generalization of the image tampering localization network model for the ACDSee tampered image, which plays an important role in overcoming the limitation that the current research in the field of image tampering localization only focuses on images tampered with by Photoshop and Deepfake. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of a preferred embodiment of the ACDSee image tampering location method based on a large model in the present invention;

[0039] Figure 2 It is a schematic diagram of an ACDSee tampered image and its corresponding tampered mark image disclosed by the present invention;

[0040] Figure 3 It is a schematic diagram of the distribution of the number of ACDSee tampered images disclosed in the present invention;

[0041] Figure 4 It is a schematic diagram of an image tampering localization network model based on a large model encoder with U-Net as the backbone network disclosed in the present invention;

[0042] Figure 5 It is a schematic diagram of an image tampering localization network model based on a large model encoder with DFCN as the backbone network disclosed in the present invention;

[0043] Figure 6 It is a schematic diagram of a parallel structure of a single feature extraction block and a low-rank adapter disclosed in the present invention;

[0044] Figure 7 It is a schematic diagram showing the positioning of the tampered area of ​​the image tampering positioning network model of the present application and other image tampering positioning networks disclosed in the present invention on the actual ACDSee tampered image;

[0045] Figure 8 It is a functional principle block diagram of a preferred embodiment of the image tampering positioning device based on a large model in the present invention;

[0046] Fig. 9 It is a functional principle block diagram of a preferred embodiment of the terminal in the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] See also Figure 1 , Figure 1 Flowchart of the ACDSee image tampering location method based on the large model in the present invention. Figure 1 As shown, the ACDSee image tampering location method based on a large model described in an embodiment of the present invention can be applied to a smart terminal, and the method includes:

[0049] Step S11, constructing a training data set including ACDSee tampered images and their corresponding tampered marked images.

[0050] In this embodiment, a training data set is constructed, and the training data set includes ACDSee tampered images and their corresponding tampered marked images. Specifically, the tampering tool in the ACDSee software is called by a pre-written image tampering script program, and the acquired original image is tampered according to the tampering operation, so as to obtain a training data set including the ACDSee tampered image and its corresponding tampered marked image; wherein the tampering tool and the tampering operation are the tools used and the operations performed when the actual ACDSee tampered image is constructed by the ACDSee software, and the image tampering script program is a script program pre-written based on the tampering tool and the tampering operation. It can be understood that the ACDSee tampered image and its corresponding tampered marked image are generated by the image tampering script program, and used as training samples for subsequent model training.

[0051] It should be pointed out that before writing the image tampering script program, the developer first uses the ACDSee software to construct an actual ACDSee tampered image based on prior knowledge to record the tampering tools and tampering operations used in constructing the actual ACDSee tampered image. The tampering tools and tampering operations may include AI selection of subjects, smart erasing, feathering tools, blur tools, exposure tools, refinement tools, rotation tools, flip tools, etc. For example, 500 images tampered with by ACDSee are made by ACDSee experts, and the tampering tools and tampering operations used in the tampering process are recorded. Then, the developer writes the corresponding image tampering script program based on the recorded tampering tools and tampering operations. In order to realize the automatic generation of ACDSee tampered images, the image tampering script program can be used to call the corresponding tampering tools and tampering operations to generate a large number of ACDSee tampered images and their corresponding tampered marked images to constitute a training data set, wherein the tampered area in the ACDSee tampered image is marked in the tampered marked image.

[0052] Furthermore, the specific implementation method of calling the tampering tool in the ACDSee software through a pre-written image tampering script program and tampering the acquired original image according to the tampering operation can be divided into three processing methods according to the usage of the "Smart Eraser Tool" in the ACDSee software, namely, a method of not using the "Smart Eraser Tool", a method of using the "Smart Eraser Tool" to erase and tamper, and a method of using the "Smart Eraser Tool" to draw the image.

[0053] The first specific implementation method is to use the "smart erase tool" to perform erasure tampering, that is, to smear and tamper with any position on the acquired original image. It can be understood that for the method of using the "smart erase tool" to perform erasure and tampering, the "smart erase tool" in the ACDSee tampering tool is called by the image tampering script program to erase and tamper with random positions in the original image, thereby generating an ACDSee tampered image and its corresponding tampered mark image.

[0054] The second specific implementation method is to use the "smart erase tool" to perform image external drawing, that is, to expand the corresponding blank area in any direction of the acquired original image, and generate the tampered image in the blank area. It can be understood that for the method of using the "smart erase tool" to perform image external drawing, the blank area of ​​the original image is first expanded in a certain direction, and then the "smart erase tool" is used on the blank area to generate the picture content, thereby generating the ACDSee tampered image and its corresponding tampered mark image.

[0055] The third specific implementation method does not use the "smart erase tool", that is, determining the original foreground image and the original background image of the acquired original image, and splicing any area in the original foreground image to any position in the original background image. It can be understood that this method requires obtaining the original foreground image and the original background image corresponding to the original image, and then calling the tampering tool through the image tampering script program to splice a certain area in the original foreground image to a random position in the original background image, thereby generating an ACDSee tampered image and its corresponding tampered mark image.

[0056] It should also be pointed out that public datasets such as UCID, Vision, and Dresden can be used as original images. For example, 7565 images in Vision and 4570 images taken by other experts are obtained as original images. Then, the tampering tool in ACDSee software is called through the image tampering script program, and these original images are tampered with according to the tampering operation to obtain the corresponding ACDSee tampered images and their corresponding tampered marked images, such as Figure 2 As shown in the figure, the image tampering types in these training samples include erasure, splicing, copy and move, image external drawing, splicing and external drawing, copy and move and image external drawing, etc. The number of each tampering type in the training samples is distributed as shown in the figure. Figure 3 shown.

[0057] Step S12: construct an image tampering localization network model based on a preset large model encoder and a preset backbone network.

[0058] In this embodiment, a dual-stream image tampering localization network model based on a preset large model encoder is constructed, that is, the image tampering localization network model is constructed based on the preset large model encoder and the preset backbone network. In other words, a dual-stream image tampering localization network model based on the large model encoder is constructed by combining the pre-trained encoder in the image segmentation large model with the backbone network.

[0059] It can be understood that the image tampering localization network model is composed of a preset backbone network and a preset large model encoder branch, wherein the preset backbone network is a network of an encoder and decoder structure that can execute image segmentation tasks, the preset large model encoder is a pre-trained encoder in the image segmentation large model based on the encoder and decoder structure, the encoder of the preset backbone network is connected to the preset large model encoder, and the encoder of the preset backbone network mainly extracts local features, and the preset large model encoder mainly extracts global features, that is, the large model encoder and the encoder of the backbone network are combined to constitute a feature extraction module in the image tampering localization network model, and the decoder of the backbone network constitutes a positioning decoding module in the image tampering localization network model, the positioning decoding module is connected to the feature extraction module, and the positioning decoding module is used to generate the result of locating the tampered area.

[0060] It should be pointed out that the preset large model encoder includes multiple feature extraction blocks connected in sequence and each feature extraction block is connected in parallel with a low-rank adapter. The feature extraction block is a transformer block that supports window attention and residual connection; wherein, the output of the feature extraction block is the result of processing the input through the window attention mechanism, adding the input to the original input through a residual connection, and then processing it through a multi-layer perceptron; the low-rank adapter is used to replace the original weight matrix change with the product of two low-rank matrices, and update the weight variables suitable for the image tampering localization task into the weight matrix.

[0061] In addition, the feature extraction block includes a local attention feature extraction block and a global attention feature extraction block; wherein the local attention feature extraction block is a feature extraction block that uses local window attention, and the size of the local window is a preset patch size in the image tampering localization network model; the global attention feature extraction block is a feature extraction block that uses global window attention, and the size of the global window is the image size input to the image tampering localization network model.

[0062] In this embodiment, the number of feature extraction blocks can be an integer multiple of 3, and the feature extraction blocks can be connected in sequence with three feature extraction blocks as a group, which are respectively two local attention feature extraction blocks and one global attention feature extraction block. That is to say, every three feature extraction blocks are a group, which are respectively two local attention feature extraction blocks and one global attention feature extraction block.

[0063] For example, using U-Net as the backbone network or DFCN as the backbone network, and using the encoder of the SAM (Segment Anything Model) image segmentation model to build an image tampering localization network model, where U-Net is used as the backbone network. Figure 4 As shown, DFCN is used as the backbone network. Figure 5 As shown, the encoder of the SAM image segmentation model includes 12 feature extraction blocks, each of which is connected in parallel to a trainable low-rank adapter. The parallel structure of a single feature extraction block and a low-rank adapter is as follows: Figure 6 As shown, the dark modules are low-rank adapter modules connected in parallel.

[0064] Step S13: Use the training data set to train the image tampering localization network model to obtain a trained image tampering localization network model.

[0065] In this embodiment, the ACDSee tampered images and their corresponding tampered marked images in the training data set are used as training samples and input into the image tampering localization network model for training, that is, the backbone network is pre-trained using the training data set and the large model encoder is fine-tuned.

[0066] It can be understood that the image tampering localization network model based on the large model encoder, with the help of the knowledge learned from the image segmentation large model, and through the ACDSee image tampering localization dataset (including ACDSee tampered images and their corresponding tampering marked images The training data set) fine-tunes the large model encoder and trains the backbone network branch. That is, the image tampering localization network model performs supervised learning under the training samples to learn the tampering traces left by the ACDSee software when editing pictures, so that the large model encoder is suitable for the tampering area localization task of ACDSee images downstream of the image segmentation field, thereby obtaining a network model with the ability to locate tampered areas at the pixel level.

[0067] It should be noted that the introduction of large image segmentation models has brought new inspiration to image segmentation tasks based on deep learning. Deep learning models with large parameters and complex structures trained on large-scale datasets can show detection performance and generalization far exceeding conventional models. The field of medical image segmentation has also applied large models to downstream tasks of image segmentation through fine-tuning and other means.

[0068] Specifically, the weights corresponding to the feature extraction blocks in the pre-loaded large model encoder are frozen; the training data set is input into the image tampering localization network model for training to update the weights of the non-frozen areas, and a trained image tampering localization network model is obtained; wherein the weights corresponding to the feature extraction blocks are the weights obtained by training the large image segmentation model on the image segmentation data set, and the non-frozen areas include the encoder and decoder of the backbone network and the low-rank adapter.

[0069] It can be understood that after the image tampering localization network model is constructed, the weights of the image segmentation large model trained on the image segmentation dataset, that is, the weights of the feature extraction block in the large model encoder, are loaded to the large model encoder, and the weights of the feature extraction block are frozen. In the image tampering localization network model, the area where the image segmentation large model pre-trained weights are loaded to the large model encoder is the frozen area, and then the ACDSee tampered images and their corresponding tampered marked images in the training dataset are used as training samples and input into the image tampering localization network model for training to update the weights of the non-frozen areas. The resulting network has the ability to locate tampered areas at the pixel level, and it is used as a trained tampering localization network.

[0070] For example, in the process of training the image tampering location network model, PyTorch 1.8.0 can be used to implement the ACDSee image tampering location method based on a large model proposed in this application, and all experiments are performed on the NVIDIA A100 GPU (a graphics card). Among them, PyTorch is an open source Python machine learning library. The ACDSee tampered images generated by the image tampering script program and their corresponding tampered marked images are used as training samples, such as using three tampering methods of erasing, splicing, and copying and moving to obtain pictures, and then the training samples are randomly divided into training sets and validation sets in a ratio of 9:1, and the actual ACDSee tampered images handmade by ACDSee experts are used as test sets. The test set includes original images, tampered images without image external drawing, and tampered images with image external drawing. During the training and testing stages, the input image resolution can be scaled to 512×512, using the Adam optimizer, and an adaptive decay learning rate strategy. When the loss fails to decrease in 2 epochs, the learning rate decays by 5%, the batch size is 32, and the initial learning rate can be set to 1×10 -4 , the epochs in the training phase can be set to 200, and AUC, F1 and IoU are used as evaluation indicators of model performance.

[0071] Step S14: input the image to be tested into the trained image tampering location network model to identify and locate the tampered area of ​​the image to be tested, and obtain a corresponding tampered area location result.

[0072] In this embodiment, the image tampering localization network model is trained using a training data set. After the trained image tampering localization network model is obtained, the image to be tested is input into the trained image tampering localization network model to identify and locate the tampered area of ​​the image to be tested, and the corresponding tampering area localization result is obtained, thereby achieving accurate positioning of the tampered area on the ACDSee tampered image.

[0073] It can be seen that in the embodiment of the present invention, the ACDSee tampered images and their corresponding tampered marked images in the constructed training data set are used as training samples, and then the image tampering localization network model constructed based on the preset large model encoder and the preset backbone network is trained using the training samples, so as to obtain a network model that can accurately locate the tampered area on the ACDSee tampered image, that is, the trained image tampering localization network model can accurately locate the editing behavior on the ACDSee tampered image, and thus can greatly improve the generalization of the image tampering localization network model for the ACDSee tampered image, that is, the trained image tampering localization network model shows excellent performance on the tampered images edited by the ACDSee software, which plays an important role in overcoming the limitation that the current research in the field of image tampering localization only focuses on images tampered with by Photoshop and Deepfake.

[0074] For example, the effectiveness of the training data set generation method proposed in the present invention can be verified by comparing some currently disclosed image tampering localization models, such as OSN, MVSS and DFCN, that is, the performance of the training samples generated by the pre-written image tampering script program before and after pre-training and fine-tuning on the actual ACDSee tampered image test set. The relevant test results are shown in Table 1 below:

[0075] Table 1

[0076]

[0077] Among them, the public weights are the weights obtained by training the model on public data sets. These public training data sets are edited by Photoshop software or synthesized by the OpenCV library in Python.

[0078] It can be seen from Table 1 above that the model trained on the Photoshop dataset cannot show satisfactory generalization for the images tampered with by ACDSee, but after pre-training or fine-tuning on the ACDSee dataset, the detection performance of ACDSee tampered images is greatly improved.

[0079] In addition, considering that the image painting tampering form has not appeared in the existing public data sets, the MVSS network can also be used to perform separate detection on tampered images using the image painting tampering form. The experimental results are shown in Table 2:

[0080] Table 2

[0081]

[0082]

[0083] As can be seen from Table 2 above, the model weights obtained by the author's public training on the Photoshop dataset are difficult to detect ACDSee tampered images using image external painting, but the model weights obtained by training on the ACDSee script dataset (including the training dataset containing ACDSee tampered images and their corresponding tampered marked images) show good detection performance for ACDSee tampered images using image external painting. Although the ACDSee script dataset does not use ACDSee tampered images using image external painting, the tampering operation of image external painting is essentially another way to use the "smart erase" function, so it can be detected by the model pre-trained or fine-tuned on the ACDSee script dataset.

[0084] In addition, by comparing the ACDSee image tampering location method based on a large model proposed in this application with the current tampering location method, it can be verified that the method proposed in this invention has better performance than the current tampering location method. The publicly available model with weights is fine-tuned on the data set generated by the script, and the model without weights is pre-trained on the data set generated by the script. The trained model is tested on the actual ACDSee tampering image data set hand-made by experts. The test results are shown in Table 3:

[0085] Table 3

[0086]

[0087] From Table 3 above, we can see that the method proposed in the present invention shows better detection performance for ACDSee tampered images compared with other tampering localization methods and methods of fine-tuning large models. For comparison of detection results of some samples, see Figure 7 shown.

[0088] In one embodiment, if Figure 8 As shown, based on the above-mentioned ACDSee image tampering positioning method based on a large model, the present invention also provides an image tampering positioning device based on a large model, including:

[0089] A data set construction module 11 is used to construct a training data set including ACDSee tampered images and their corresponding tampered marked images;

[0090] A model building module 12, used to build an image tampering positioning network model based on a preset large model encoder and a preset backbone network;

[0091] A model training module 13 is used to train the image tampering localization network model using the training data set to obtain a trained image tampering localization network model;

[0092] The tampered area positioning module 14 is used to input the image to be tested into the trained image tampering positioning network model to identify and locate the tampered area of ​​the image to be tested, and obtain the corresponding tampered area positioning result.

[0093] Fig. 9 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may include:

[0094] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0095] When the processor 502 executes the program, the ACDSee image tampering location method based on the large model provided in the above embodiment is implemented.

[0096] Furthermore, the terminal further includes:

[0097] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0098] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0099] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0100] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0101] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0102] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0103] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the ACDSee image tampering location method based on a large model as described above is implemented.

[0104] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0105] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0106] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes additional implementations, in which the order shown or discussed may not be followed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0107] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can read instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary and then storing it in a computer memory.

[0108] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0109] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0110] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0111] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

[0112] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A large-scale model-based ACDSee image tampering location method, characterized in that: The method comprises: Construct a training dataset containing ACDSee tampered images and their corresponding tampered marked images; Construct an image tampering localization network model based on a preset large model encoder and a preset backbone network; Using the training data set to train the image tampering localization network model to obtain a trained image tampering localization network model; The image to be tested is input into the trained image tampering location network model to identify and locate the tampered area of ​​the image to be tested, and obtain the corresponding tampered area location result.

2. The ACDSee image tampering location method based on a large model according to claim 1 is characterized in that: The construction includes a training data set of ACDSee tampered images and their corresponding tampered marked images, including: The tampering tool in the ACDSee software is called by a pre-written image tampering script program, and the acquired original image is tampered with according to the tampering operation, so as to obtain a training data set including the ACDSee tampered image and its corresponding tampered marked image; The tampering tool and the tampering operation are the tools used and the operations performed when the actual ACDSee tampering image constructed by the ACDSee software, and the image tampering script program is a script program pre-written based on the tampering tool and the tampering operation.

3. The ACDSee image tampering location method based on a large model according to claim 2 is characterized in that: The tampering of the acquired original image includes: Smear and tamper with any position on the acquired original image; Or, expanding a corresponding blank area in any direction in the acquired original image, and generating a tampered image in the blank area; Alternatively, an original foreground image and an original background image of the acquired original image are determined, and any area in the original foreground image is spliced ​​to any position in the original background image.

4. The ACDSee image tampering location method based on a large model according to any one of claims 1 to 3, characterized in that: The preset backbone network is a network of an encoder and decoder structure that can perform image segmentation tasks, the preset large model encoder is an encoder pre-trained in an image segmentation large model based on the encoder and decoder structure, and the encoder of the preset backbone network for extracting local features is interconnected with the preset large model encoder for extracting global features.

5. The ACDSee image tampering location method based on a large model according to claim 4 is characterized in that: The preset large model encoder includes a plurality of feature extraction blocks connected in sequence and each of the feature extraction blocks is connected in parallel with a low-rank adapter, and the feature extraction block is a transformer block supporting window attention and residual connection; Furthermore, the feature extraction block includes a local attention feature extraction block and a global attention feature extraction block; Wherein, the local attention feature extraction block is a feature extraction block using local window attention, and the size of the local window is a preset patch size in the image tampering localization network model; The global attention feature extraction block is a feature extraction block using global window attention, and the size of the global window is the image size input to the image tampering localization network model.

6. The ACDSee image tampering location method based on a large model according to claim 5 is characterized in that: The number of the feature extraction blocks is an integer multiple of three, and the feature extraction blocks are connected in sequence in such a way that three feature extraction blocks form a group, and the three feature extraction blocks are respectively two local attention feature extraction blocks and one global attention feature extraction block.

7. The ACDSee image tampering location method based on a large model according to claim 6 is characterized in that: The method of training the image tampering localization network model using the training data set to obtain a trained image tampering localization network model includes: Freeze the weights corresponding to the feature extraction blocks in the pre-loaded large model encoder; Inputting the training data set into the image tampering localization network model for training to update the weights of the non-frozen areas, thereby obtaining a trained image tampering localization network model; The weights corresponding to the feature extraction blocks are the weights obtained by training the large image segmentation model on an image segmentation data set, and the non-frozen area includes the encoder and decoder of the backbone network and the low-rank adapter.

8. A large-model-based image tampering location device, characterized in that: The device comprises: A data set construction module, used to construct a training data set including ACDSee tampered images and their corresponding tampered marked images; A model building module, used to build an image tampering localization network model based on a preset large model encoder and a preset backbone network; A model training module, used to train the image tampering localization network model using the training data set to obtain a trained image tampering localization network model; The tampered area positioning module is used to input the image to be tested into the trained image tampering positioning network model to identify and locate the tampered area of ​​the image to be tested, and obtain the corresponding tampered area positioning result.

9. A terminal, characterized in that: include: A memory, a processor, and a large model-based image tampering location program stored in the memory and executable on the processor, wherein the large model-based image tampering location program, when executed by the processor, implements the steps of the large model-based ACDSee image tampering location method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which can be executed to implement the steps of the ACDSee image tampering positioning method based on a large model as described in any one of claims 1 to 7.

Images