ACDSee image tampering positioning method, device, terminal and medium based on large model

Abstract

This invention provides a method, apparatus, terminal, and medium for ACDSee image tampering localization based on a large model. The method includes: constructing a training dataset containing ACDSee tampered images and their corresponding tamper-marked images; constructing an image tampering localization network model based on a preset large model encoder and a preset backbone network; training the image tampering localization network model using the training dataset; and inputting a test image into the trained image tampering localization network model to identify and locate the tampered region in the test image, thereby obtaining the tampering region localization result. This invention utilizes the constructed training dataset containing ACDSee tampered images and their corresponding tamper-marked images to train the constructed image tampering localization network model, thereby obtaining a network model capable of accurately locating tampered regions on ACDSee tampered images, improving the generalization ability of the image tampering localization network model for ACDSee tampered images.

Description

Technical Field

[0001] This invention relates to the field of multimedia information security, and in particular to a method, apparatus, terminal and medium for locating ACDSee image tampering based on a large model. Background Technology

[0002] In recent years, with the development of artificial intelligence technology, the operational threshold for image editing has become increasingly lower. Driven by ulterior motives, some criminals tamper with digital images for profit. These altered images are widely disseminated through social media, posing a significant threat to social stability and personal privacy.

[0003] ACDSee, a popular image browsing and editing software, was primarily used for image browsing and management in its earlier versions. However, in recent years, with the continuous improvement of its editing functions and the integration of AI features, users can easily use ACDSee to edit and tamper with image content. Given ACDSee's large user base, there is a potential issue of large numbers of images modified using ACDSee circulating online, causing serious harm.

[0004] Currently, although extensive research has been conducted in the field of image forensics, most of it has been focused on images tampered with by Photoshop or Deepfake. Due to the different algorithms and processing techniques used, ACDSee exhibits different characteristics in image editing. Existing deep learning-based image tampering localization methods cannot show satisfactory generalization performance on images tampered with by ACDSee software.

[0005] In summary, existing deep learning-based image tampering localization techniques exhibit poor generalization performance against images edited and tampered with by ACDSee software. Therefore, providing a solution to these technical problems is a current challenge for those skilled in the art. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method, device, terminal and medium for locating ACDSee image tampering based on a large model, in order to address the above-mentioned deficiencies of the prior art. The aim is to solve the problem of poor generalization of existing image tampering location technologies for ACDSee edited and tampered images.

[0007] The technical solution adopted by this invention to solve the technical problem is as follows:

[0008] A method for locating ACDSee image tampering based on a large model, wherein the method includes:

[0009] Construct a training dataset containing ACDSee tampered images and their corresponding tampered marked images;

[0010] An image tampering localization network model is constructed based on a pre-set large model encoder and a pre-set backbone network.

[0011] The image tampering localization network model is trained using the training dataset to obtain a trained image tampering localization network model;

[0012] The image to be tested is input into the trained image tampering localization network model to identify and locate the tampered area of ​​the image to be tested, and the corresponding tampering area localization result is obtained.

[0013] In one implementation, constructing a training dataset containing ACDSee tampered images and their corresponding tamper-marked images includes:

[0014] The pre-written image tampering script calls the tampering tool in the ACDSee software and tampers with the original image according to the tampering operation, resulting in a training dataset containing ACDSee tampered images and their corresponding tampered marked images.

[0015] The tampering tool and the tampering operation are the tools and operations used when actually tampering with an ACDSee image constructed by the ACDSee software, and the image tampering script program is a script program pre-written based on the tampering tool and the tampering operation.

[0016] In one implementation, the alteration of the acquired original image includes:

[0017] The original image was altered by smearing or tampering at any location.

[0018] Alternatively, a corresponding blank area can be extended in any direction in the acquired original image, and a tampered image can be generated in the blank area;

[0019] Alternatively, determine the original foreground image and the original background image of the acquired original image, and stitch any region in the original foreground image to any position in the original background image.

[0020] In one implementation, the preset backbone network is a network capable of performing image segmentation tasks with an encoder and decoder structure, the preset large model encoder is a pre-trained encoder based on the image segmentation large model with the encoder and decoder structure, and the encoder of the preset backbone network used for extracting local features is interconnected with the preset large model encoder used for extracting global features.

[0021] In one implementation, the preset large model encoder includes multiple sequentially connected feature extraction blocks, and each feature extraction block is connected in parallel with a low-rank adapter. The feature extraction blocks are transformer blocks that support window attention and residual connections.

[0022] Furthermore, the feature extraction block includes a local attention feature extraction block and a global attention feature extraction block;

[0023] Wherein, the local attention feature extraction block is a feature extraction block using local window attention, and the size of the local window is the preset patch size in the image tampering localization network model;

[0024] The global attention feature extraction block is a feature extraction block that uses global window attention, and the size of the global window is the size of the image input to the image tampering localization network model.

[0025] In one implementation, the number of feature extraction blocks is an integer multiple of three, and the feature extraction blocks are connected in a group of three, wherein the three feature extraction blocks are two local attention feature extraction blocks and one global attention feature extraction block.

[0026] In one implementation, training the image tampering localization network model using the training dataset to obtain a trained image tampering localization network model includes:

[0027] Freeze the weights corresponding to the feature extraction blocks in the pre-loaded large model encoder;

[0028] The training dataset is input into the image tampering localization network model for training to update the weights of the non-frozen regions, thereby obtaining the trained image tampering localization network model.

[0029] The weights corresponding to the feature extraction blocks are the weights obtained by training the large image segmentation model on the image segmentation dataset, and the non-frozen regions include the encoder and decoder of the backbone network and the low-rank adapter.

[0030] The present invention also discloses an image tampering localization device based on a large model, wherein the device comprises:

[0031] The dataset construction module is used to build a training dataset containing ACDSee tampered images and their corresponding tampered marked images;

[0032] The model building module is used to build an image tampering localization network model based on a preset large model encoder and a preset backbone network.

[0033] The model training module is used to train the image tampering localization network model using the training dataset to obtain a trained image tampering localization network model.

[0034] The tampering region localization module is used to input the image to be tested into the trained image tampering localization network model to identify and locate the tampered region of the image to be tested, and obtain the corresponding tampering region localization result.

[0035] The present invention also discloses a terminal, comprising: a memory, a processor, and a large-model-based image tampering location program stored in the memory and executable on the processor, wherein the large-model-based image tampering location program, when executed by the processor, implements the steps of the large-model-based ACDSee image tampering location method as described above.

[0036] The present invention also discloses a computer-readable storage medium storing a computer program that can be executed to implement the steps of the large-model-based ACDSee image tampering localization method as described above.

[0037] This invention provides a method, apparatus, terminal, and medium for ACDSee image tampering localization based on a large model. The method includes: constructing a training dataset containing ACDSee tampered images and their corresponding tampered marker images; constructing an image tampering localization network model based on a preset large model encoder and a preset backbone network; training the image tampering localization network model using the training dataset to obtain a trained image tampering localization network model; and inputting a test image into the trained image tampering localization network model to identify and locate the tampered region of the test image, thereby obtaining the corresponding tampered region localization result. Therefore, this invention uses ACDSee tampered images and their corresponding tampered marker images in the constructed training dataset as training samples. Then, it uses these training samples to train an image tampering localization network model constructed based on a preset large model encoder and a preset backbone network. This results in a network model that can accurately locate the tampered region on ACDSee tampered images. In other words, the trained image tampering localization network model can accurately locate editing behavior on ACDSee tampered images, thereby greatly improving the generalization ability of the image tampering localization network model for ACDSee tampered images. This plays an important role in overcoming the limitation of current research in the field of image tampering localization, which only focuses on images tampered with by Photoshop and Deepfake. Attached Figure Description

[0038] Figure 1 This is a flowchart of a preferred embodiment of the ACDSee image tampering localization method based on a large model in this invention;

[0039] Figure 2 This is a schematic diagram of an ACDSee tampered image and its corresponding tampered mark image disclosed in this invention;

[0040] Figure 3 This is a schematic diagram of the distribution of the number of tampered images disclosed in this invention;

[0041] Figure 4 This is a schematic diagram of an image tampering localization network model based on a large model encoder with U-Net as the backbone network disclosed in this invention;

[0042] Figure 5 This is a schematic diagram of an image tampering localization network model based on a large model encoder with DFCN as the backbone network disclosed in this invention;

[0043] Figure 6 This is a schematic diagram of the parallel structure of a single feature extraction block and a low-rank adapter disclosed in this invention;

[0044] Figure 7 This is a schematic diagram showing the image tampering localization network model disclosed in this application and other image tampering localization networks in the localization performance of tampered areas on actual ACDSee tampered images;

[0045] Figure 8 This is a functional principle block diagram of a preferred embodiment of the image tampering location device based on a large model in this invention;

[0046] Figure 9 This is a functional principle block diagram of a preferred embodiment of the terminal in this invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0048] See Figure 1 , Figure 1 This is a flowchart of the ACDSee image tampering localization method based on a large model in this invention. Figure 1 As shown in the embodiments of the present invention, the ACDSee image tampering localization method based on a large model can be applied to smart terminals. The method includes:

[0049] Step S11: Construct a training dataset containing ACDSee tampered images and their corresponding tampered marked images.

[0050] In this embodiment, a training dataset is constructed, which contains ACDSee tampered images and their corresponding tamper-marked images. Specifically, a pre-written image tampering script program calls the tampering tools in the ACDSee software and tampers with the acquired original images according to the tampering operations, resulting in a training dataset containing ACDSee tampered images and their corresponding tamper-marked images. The tampering tools and tampering operations refer to the tools and operations used when constructing actual ACDSee tampered images using the ACDSee software, and the image tampering script program is a pre-written script program based on the tampering tools and tampering operations. It can be understood that the image tampering script program generates ACDSee tampered images and their corresponding tamper-marked images, which are then used as training samples for subsequent model training.

[0051] It should be noted that before writing the image manipulation script, developers first use ACDSee software to construct actual ACDSee manipulated images based on prior knowledge. This process records the manipulation tools and operations used in constructing the actual ACDSee manipulated images. These manipulation tools and operations can include AI subject selection, intelligent erasing, feathering, blurring, exposure, thinning, rotation, and flipping tools. For example, 500 ACDSee manipulated images can be created using ACDSee experts, and the manipulation tools and operations used during the manipulation process can be recorded. Then, developers write corresponding image manipulation scripts based on the recorded manipulation tools and operations. In order to automatically generate ACDSee manipulated images, the image manipulation script can call the corresponding manipulation tools and operations to generate a large number of ACDSee manipulated images and their corresponding manipulated marker images, forming a training dataset. The manipulated areas in the ACDSee manipulated images are marked in the manipulated images.

[0052] Furthermore, the specific implementation method of calling the tampering tool in ACDSee software through a pre-written image tampering script program and tampering the acquired original image according to the tampering operation can be divided into three processing methods according to the usage of the "intelligent erase tool" in ACDSee software: the method of not using the "intelligent erase tool", the method of erasing and tampering using the "intelligent erase tool", and the method of using the "intelligent erase tool" for image drawing.

[0053] The first specific implementation method uses the "Smart Eraser Tool" for erasing and tampering, which involves smearing and altering any location on the acquired original image. It can be understood that, for the "Smart Eraser Tool" method, the image tampering script program calls the "Smart Eraser Tool" in the ACDSee tampering tool to erase and tamper with random locations in the original image, thereby generating an ACDSee tampered image and its corresponding tampering mark image.

[0054] The second specific implementation method uses an "intelligent eraser tool" for image outlining. This involves extending a corresponding blank area in any direction within the acquired original image and generating a tampered image within that blank area. Essentially, the method using the "intelligent eraser tool" first extends a blank area in a certain direction from the original image, then uses the "intelligent eraser tool" to generate image content within that blank area, thus generating the ACDSee tampered image and its corresponding tamper mark image.

[0055] The third specific implementation method does not use the "smart erasure tool." Instead, it identifies the original foreground and background images of the acquired original image and stitches any region from the original foreground image to any position in the original background image. Understandably, this method requires obtaining the original foreground and background images corresponding to the original image, and then using an image tampering script to call a tampering tool to stitch a region from the original foreground image to a random position in the original background image, thereby generating an ACDSee tampered image and its corresponding tampering mark image.

[0056] It should also be noted that publicly available datasets such as UCID, Vision, and Dresden can be used as the original images. For example, 7565 images from Vision and 4570 images taken by other experts can be used as the original images. Then, an image manipulation script can be used to call the manipulation tool in ACDSee software and manipulate these original images according to the manipulation operation to obtain the corresponding ACDSee manipulated images and their corresponding manipulated marked images, such as... Figure 2 As shown, these were used as training samples. The types of image manipulation in these training samples included erasure, splicing, copying and moving, image outlining, splicing and outlining, copying and moving, and image outlining. The distribution of the number of each manipulation type in the training samples is as follows: Figure 3 As shown.

[0057] Step S12: Construct an image tampering localization network model based on a preset large model encoder and a preset backbone network.

[0058] In this embodiment, a dual-stream image tampering localization network model based on a preset large-scale encoder is constructed. That is, the image tampering localization network model is constructed based on a preset large-scale encoder and a preset backbone network. In other words, the dual-stream image tampering localization network model based on a large-scale encoder is constructed by combining the pre-trained encoder in the image segmentation large-scale model with the backbone network.

[0059] It is understandable that the image tampering localization network model consists of a preset backbone network and a preset large-scale encoder branch. The preset backbone network is a network with an encoder and decoder structure capable of performing image segmentation tasks. The preset large-scale encoder is a pre-trained encoder in a large-scale image segmentation model based on an encoder and decoder structure. The encoder of the preset backbone network and the encoder of the preset large-scale encoder are interconnected. The encoder of the preset backbone network mainly extracts local features, while the encoder of the preset large-scale encoder mainly extracts global features. That is, the combination of the large-scale encoder and the encoder of the backbone network constitutes the feature extraction module in the image tampering localization network model. The decoder of the backbone network constitutes the localization decoding module in the image tampering localization network model. The localization decoding module is connected to the feature extraction module and is used to generate the result of tampering region localization.

[0060] It should be noted that the preset large model encoder includes multiple sequentially connected feature extraction blocks, and each feature extraction block is connected in parallel with a low-rank adapter. The feature extraction blocks are transformer blocks that support window attention and residual connections. The output of the feature extraction block is the result obtained by adding the input processed by the window attention mechanism to the original input through residual connections, and then processing it through a multilayer perceptron. The low-rank adapter is used to update the weight variables suitable for image tampering localization tasks into the weight matrix by replacing the changes in the original weight matrix with the product of two low-rank matrices.

[0061] Furthermore, the feature extraction block includes a local attention feature extraction block and a global attention feature extraction block; wherein, the local attention feature extraction block is a feature extraction block that uses local window attention, and the size of the local window is the preset patch size in the image tampering localization network model; the global attention feature extraction block is a feature extraction block that uses global window attention, and the size of the global window is the image size input to the image tampering localization network model.

[0062] In this embodiment, the number of feature extraction blocks can be an integer multiple of 3, and the feature extraction blocks can be connected in a group of three, with each group consisting of two local attention feature extraction blocks and one global attention feature extraction block. In other words, every three feature extraction blocks are grouped together, consisting of two local attention feature extraction blocks and one global attention feature extraction block.

[0063] For example, an image tampering localization network model can be constructed using U-Net or DFCN as the backbone network and the encoder of a large-scale image segmentation model (SAM, Segment Anything Model) as the backbone network. See [link to U-Net example]. Figure 4 As shown, the backbone network is DFCN. (See attached image) Figure 5 As shown, the encoder of the SAM image segmentation large model includes 12 feature extraction blocks, each of which is connected in parallel to a trainable low-rank adapter. The parallel structure of a single feature extraction block and the low-rank adapter is as follows. Figure 6 As shown, the dark-colored modules are low-rank adapter modules connected in parallel.

[0064] Step S13: Train the image tampering localization network model using the training dataset to obtain the trained image tampering localization network model.

[0065] In this embodiment, the ACDSee tampered images and their corresponding tampered marked images in the training dataset are used as training samples and input into the image tampering localization network model for training. That is, the backbone network is pre-trained using the training dataset, and the large model encoder is fine-tuned.

[0066] Understandably, the image tampering localization network model based on the large model encoder leverages knowledge learned from the large image segmentation model and fine-tunes the large model encoder and trains the backbone network branches using the ACDSee image tampering localization dataset (a training dataset containing ACDSee tampered images and their corresponding tampering-marked images). In other words, the image tampering localization network model undergoes supervised learning under training samples, learning the tampering traces left when editing images with ACDSee software. This makes the large model encoder suitable for the downstream task of ACDSee image tampering region localization in the field of image segmentation, thus obtaining a network model with pixel-level tampering region localization capabilities.

[0067] It should be noted that the proposal of large-scale image segmentation models has brought new inspiration to deep learning-based image segmentation tasks. Deep learning models trained on large-scale datasets with large-scale parameters and complex structures can exhibit detection performance and generalization ability far exceeding those of conventional models. In the field of medical image segmentation, large-scale models have also been applied to downstream tasks of image segmentation through fine-tuning and other means.

[0068] Specifically, the weights corresponding to the feature extraction blocks in the pre-loaded large model encoder are frozen; the training dataset is input into the image tampering localization network model for training to update the weights of the non-frozen regions, resulting in a trained image tampering localization network model; wherein, the weights corresponding to the feature extraction blocks are the weights obtained by training the large image segmentation model on the image segmentation dataset, and the non-frozen regions include the encoder and decoder of the backbone network as well as the low-rank adapter.

[0069] Understandably, after the image tampering localization network model is built, the weights obtained by the large image segmentation model trained on the image segmentation dataset are loaded onto the large model encoder, which are the weights of the feature extraction blocks in the large model encoder, and the weights of the feature extraction blocks are frozen. In the image tampering localization network model, the region where the pre-trained weights of the image segmentation large model are loaded onto the large model encoder is the frozen region. Then, the ACDSee tampered images and their corresponding tampering-marked images in the training dataset are used as training samples and input into the image tampering localization network model to update the weights of the non-frozen regions. The resulting network with pixel-level tampering region localization capability is used as the trained tampering localization network.

[0070] For example, in training the image tampering localization network model, PyTorch 1.8.0 can be used to implement the proposed large-model-based ACDSee image tampering localization method, and all experiments are conducted on an NVIDIA A100 GPU (a type of graphics card). PyTorch is an open-source Python machine learning library. ACDSee tampered images and their corresponding tampering-marked images generated by an image tampering script are used as training samples. Images are obtained using three tampering methods: erasure, stitching, and copying / moving. The training samples are then randomly divided into training and validation sets at a 9:1 ratio. Actual ACDSee tampered images manually created by ACDSee experts are used as the test set, which includes the original image, tampered images without image rendering, and tampered images with image rendering. During training and testing, the input image resolution can be scaled to 512×512. The Adam optimizer is used with an adaptive decay learning rate strategy. When the loss fails to decrease within two epochs, the learning rate decays by 5%. The batch size is 32, and the initial learning rate can be set to 1×10⁻⁶. -4 The number of epochs during the training phase can be set to 200, and AUC, F1, and IoU can be used as evaluation metrics for model performance.

[0071] Step S14: Input the image to be tested into the trained image tampering localization network model to identify and locate the tampered area of ​​the image to be tested, and obtain the corresponding tampering area localization result.

[0072] In this embodiment, the image tampering localization network model is trained using a training dataset. After obtaining the trained image tampering localization network model, the image to be tested is input into the trained image tampering localization network model to identify and locate the tampered area of ​​the image to be tested, and obtain the corresponding tampered area localization result, thereby realizing the accurate localization of the tampered area on the ACDSee tampered image.

[0073] As can be seen, in this embodiment of the invention, by using the ACDSee tampered images and their corresponding tampered marked images in the constructed training dataset as training samples, and then using these training samples to train the image tampering localization network model constructed based on a preset large model encoder and a preset backbone network, a network model that can accurately locate the tampered region on the ACDSee tampered image is obtained. That is, the trained image tampering localization network model can accurately locate the editing behavior on the ACDSee tampered image, thereby greatly improving the generalization of the image tampering localization network model to ACDSee tampered images. In other words, the trained image tampering localization network model exhibits excellent performance on tampered images edited by ACDSee software, which plays an important role in overcoming the limitation of current research in the field of image tampering localization that only focuses on images tampered by Photoshop and Deepfake.

[0074] For example, the effectiveness of the training dataset generation method proposed in this invention can be verified by comparing it with some currently available image tampering localization models, such as OSN, MVSS, and DFCN. Specifically, the performance of the training samples generated by the pre-written image tampering script program before and after pre-training and fine-tuning on the actual ACDSee tampered image test set can be verified. The relevant test results are shown in Table 1 below:

[0075] Table 1

[0076]

[0077] The public weights are the weights obtained by training the model on public datasets, which are either edited using Photoshop or synthesized using the OpenCV library in Python.

[0078] As can be seen from Table 1 above, the model trained on the Photoshop dataset does not show satisfactory generalization for images tampered with by ACDSee, but after pre-training or fine-tuning on the ACDSee dataset, the detection performance of tampered images by ACDSee is greatly improved.

[0079] Furthermore, considering that no image outlay tampering methods have been found in the existing public datasets, the MVSS network can be used to detect tampered images using image outlay tampering methods separately. The experimental results are shown in Table 2:

[0080] Table 2

[0081]

[0082]

[0083] As can be seen from Table 2 above, the model weights trained on the Photoshop dataset, as disclosed by the authors, have difficulty detecting ACDSee tampered images using image outlining. However, the model weights trained on the ACDSee Script dataset (which includes a training dataset containing ACDSee tampered images and their corresponding tamper-marked images) show good detection performance for ACDSee tampered images using image outlining. Although the ACDSee Script dataset does not contain ACDSee tampered images using image outlining, image outlining tampering is essentially another way of using the "smart erase" function, and therefore can be detected by models pre-trained or fine-tuned on the ACDSee Script dataset.

[0084] Furthermore, by comparing the proposed ACDSee image tampering localization method based on a large model with current tampering localization methods, it can be verified that the proposed method has better performance than current tampering localization methods. For publicly available models with provided weights, fine-tuning was performed on a script-generated dataset; for models without provided weights, pre-training was performed on a script-generated dataset. The trained models were then tested on a real ACDSee tampered image dataset manually created by experts. The test results are shown in Table 3.

[0085] Table 3

[0086]

[0087] As can be seen from Table 3 above, the method proposed in this invention exhibits superior detection performance for ACDSee tampered images compared to other tamper location methods and methods for fine-tuning large models. For comparisons of detection results for some samples, please refer to [link to Table 3]. Figure 7 As shown.

[0088] In one embodiment, such as Figure 8 As shown, based on the above-mentioned ACDSee image tampering localization method based on a large model, the present invention also provides an image tampering localization device based on a large model, comprising:

[0089] Dataset construction module 11 is used to construct a training dataset containing ACDSee tampered images and their corresponding tampered marked images;

[0090] Model building module 12 is used to build an image tampering localization network model based on a preset large model encoder and a preset backbone network;

[0091] Model training module 13 is used to train the image tampering localization network model using the training dataset to obtain a trained image tampering localization network model.

[0092] The tampering region localization module 14 is used to input the image to be tested into the trained image tampering localization network model to identify and locate the tampered region of the image to be tested, and obtain the corresponding tampering region localization result.

[0093] Figure 9 A schematic diagram of the structure of a terminal provided in an embodiment of this application. The terminal may include:

[0094] The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0095] When the processor 502 executes the program, it implements the ACDSee image tampering location method based on a large model provided in the above embodiments.

[0096] Furthermore, the terminal also includes:

[0097] Communication interface 503 is used for communication between memory 501 and processor 502.

[0098] The memory 501 is used to store computer programs that can run on the processor 502.

[0099] The memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0100] If the memory 501, processor 502, and communication interface 503 are implemented independently, they can be interconnected via a bus to communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one line is used in the diagram, but this does not imply that there is only one bus or one type of bus.

[0101] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0102] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0103] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described ACDSee image tampering location method based on a large model.

[0104] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0105] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0106] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0107] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can read and execute instructions from or in conjunction with such an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). In addition, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically by optically scanning paper or other media, then editing, interpreting or otherwise processing them as necessary, and then storing them in computer memory.

[0108] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0109] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0110] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0111] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

[0112] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for locating ACDSee image tampering based on a large model, characterized in that, The method includes: Construct a training dataset containing ACDSee tampered images and their corresponding tampered marked images; An image tampering localization network model is constructed based on a pre-set large model encoder and a pre-set backbone network. The image tampering localization network model is trained using the training dataset to obtain a trained image tampering localization network model; The image to be tested is input into the trained image tampering localization network model to identify and locate the tampered area of ​​the image to be tested, and the corresponding tampering area localization result is obtained. The preset backbone network is a network with an encoder and decoder structure that can perform image segmentation tasks. The preset large model encoder is a pre-trained encoder in the image segmentation large model based on the encoder and decoder structure. The encoder of the preset backbone network used to extract local features is interconnected with the encoder of the preset large model used to extract global features. The preset large model encoder includes multiple sequentially connected feature extraction blocks, and each feature extraction block is connected in parallel with a low-rank adapter. The feature extraction block is a transformer block that supports window attention and residual connection. Furthermore, the feature extraction block includes a local attention feature extraction block and a global attention feature extraction block; Wherein, the local attention feature extraction block is a feature extraction block using local window attention, and the size of the local window is the preset patch size in the image tampering localization network model; The global attention feature extraction block is a feature extraction block that uses global window attention, and the size of the global window is the size of the image input to the image tampering localization network model.

2. The ACDSee image tampering localization method based on a large model according to claim 1, characterized in that, The construction of the training dataset containing ACDSee tampered images and their corresponding tampered marked images includes: The pre-written image tampering script calls the tampering tool in the ACDSee software and tampers with the original image according to the tampering operation, resulting in a training dataset containing ACDSee tampered images and their corresponding tampered marked images. The tampering tool and the tampering operation are the tools and operations used when actually tampering with an ACDSee image constructed by the ACDSee software, and the image tampering script program is a script program pre-written based on the tampering tool and the tampering operation.

3. The ACDSee image tampering localization method based on a large model according to claim 2, characterized in that, The modification of the acquired original image includes: The original image was altered by smearing or tampering at any location. Alternatively, a corresponding blank area can be extended in any direction in the acquired original image, and a tampered image can be generated in the blank area; Alternatively, determine the original foreground image and the original background image of the acquired original image, and stitch any region in the original foreground image to any position in the original background image.

4. The ACDSee image tampering localization method based on a large model according to claim 1, characterized in that, The number of feature extraction blocks is an integer multiple of three, and the feature extraction blocks are connected in a group of three, which consists of two local attention feature extraction blocks and one global attention feature extraction block.

5. The ACDSee image tampering localization method based on a large model according to claim 4, characterized in that, The step of training the image tampering localization network model using the training dataset to obtain the trained image tampering localization network model includes: Freeze the weights corresponding to the feature extraction blocks in the pre-loaded large model encoder; The training dataset is input into the image tampering localization network model for training to update the weights of the non-frozen regions, thereby obtaining the trained image tampering localization network model. The weights corresponding to the feature extraction blocks are the weights obtained by training the large image segmentation model on the image segmentation dataset, and the non-frozen regions include the encoder and decoder of the backbone network and the low-rank adapter.

6. A large-model-based image tampering localization device, characterized in that, The device includes: The dataset construction module is used to build a training dataset containing ACDSee tampered images and their corresponding tampered marked images; The model building module is used to build an image tampering localization network model based on a preset large model encoder and a preset backbone network. The model training module is used to train the image tampering localization network model using the training dataset to obtain a trained image tampering localization network model. The tampering region localization module is used to input the image to be tested into the trained image tampering localization network model to identify and locate the tampered region of the image to be tested, and obtain the corresponding tampering region localization result; The preset backbone network is a network with an encoder and decoder structure that can perform image segmentation tasks. The preset large model encoder is a pre-trained encoder in the image segmentation large model based on the encoder and decoder structure. The encoder of the preset backbone network used to extract local features is interconnected with the encoder of the preset large model used to extract global features. The preset large model encoder includes multiple sequentially connected feature extraction blocks, and each feature extraction block is connected in parallel with a low-rank adapter. The feature extraction block is a transformer block that supports window attention and residual connection. Furthermore, the feature extraction block includes a local attention feature extraction block and a global attention feature extraction block; Wherein, the local attention feature extraction block is a feature extraction block using local window attention, and the size of the local window is the preset patch size in the image tampering localization network model; The global attention feature extraction block is a feature extraction block that uses global window attention, and the size of the global window is the size of the image input to the image tampering localization network model.

7. A terminal, characterized in that, include: The system includes a memory, a processor, and a large-model-based image tampering location program stored in the memory and executable on the processor. When executed by the processor, the large-model-based image tampering location program implements the steps of the large-model-based ACDSee image tampering location method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed to implement the steps of the ACDSee image tampering localization method based on a large model as described in any one of claims 1 to 5.

Images