Image tampering detection method based on visual basic model

Abstract

The invention provides an image tampering detection method based on a visual basic model, which comprises the steps of cross-view automatic prompt generation, feature extraction from various complementary views and fusion into prompts, and manual prompt dependence elimination. Selecting an optimal prompt based on the minimum segmentation loss through an optimal prompt selection module; utilizing a cross-view consistency module to enhance view prompt consistency with consistency loss; multi-view features are aligned and fused through a cross-view feature perception module; fusing the features and the optimal prompt through a prompt mixing module to generate final prompt embedding; and finally, inputting the fusion prompt and the feature into an SAM mask decoder to realize tampering detection and positioning. According to the method, the visual basic model is applied to tampering detection, and the performance is improved through multi-view processing. Experimental results show that the method significantly improves model generalization, robustness and tampering positioning precision.

Description

TECHNICAL FIELD

[0001] The present application relates to computer vision and multimedia forensics technology, and particularly to an image tampering detection method based on a visual foundation model. BACKGROUND

[0002] Currently, image tampering detection methods can be broadly divided into two categories, mainly through identifying tampering artifacts for distinction. Some techniques rely on detecting abnormal features, usually using high-pass noise filters to suppress content information. Other methods attempt to detect compression inconsistencies in tampered images, as they assume different compression quality factors before and after tampering. In addition, some researchers focus on camera-based artifacts, such as model fingerprints.

[0003] With the continuous advancement of deep learning technology, more and more image tampering detection methods begin to use deep convolutional neural networks (CNN) to automatically extract image features for tampering detection. This kind of method can automatically identify tampering features from images by learning a large amount of image data, improving detection accuracy. However, deep learning methods still face some challenges in practical applications. A) Poor generalization ability: deep learning models usually require a large amount of labeled data for training, but their generalization ability is poor for images from different datasets or domains. In practical applications, the model may perform well on one dataset but poorly on another, failing to meet the cross-dataset application requirements. B) Poor robustness: existing deep learning models often perform unstably when facing image quality changes, compression or blurring interference. Especially in high compression ratio or poor quality images, the performance of existing models may drop sharply. C) Dependence on manual hints: many existing deep learning models still require manual input of hints, such as selecting regions of interest or generating bounding boxes, which poses a challenge to the realization of fully automated image tampering detection.

[0004] In recent years, foundation models such as GPT-4, Flamingo and SAM have brought about tremendous changes in the field of artificial intelligence through large-scale pre-training. Inspired by the success of foundation models in natural language processing (NLP), researchers have begun to explore their potential applications in computer vision. SAM uniquely utilizes a large number of masks, demonstrating strong generalization ability. However, when SAM was initially designed, it was a task-independent segmentation model that required input image hints (such as points, boxes or masks), so it did not directly support end-to-end automated segmentation perception. There are significant challenges in using SAM for image tampering detection tasks. a) Dependence on manual hints: SAM's interactive framework requires predefined hints (such as points, boxes or masks) as input images; b) Single-view information is difficult to support cross-dataset generalization.

[0005] It should be noted that the information disclosed in the above background section is only for understanding the background of the present application, and therefore can include information that does not constitute the prior art known to those of ordinary skill in the art. SUMMARY

[0006] The main purpose of the present application is to overcome the defects existing in the background art, and provide an image tampering detection method based on a visual base model.

[0007] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0008] An image tampering detection method based on a visual base model, comprising the following steps:

[0009] S1, cross-view automatic prompt generation: image features are extracted through multiple complementary views, and unified prompt information is generated by fusion to eliminate manual prompt dependence;

[0010] S2, optimal prompt selection: based on minimizing segmentation loss, the optimal prompt is selected from the prompts generated from multiple views;

[0011] S3, cross Figure One consistency optimization: the consistency of prompt information of different views is enhanced through consistency loss;

[0012] S4, cross-view feature perception fusion: align and fuse multi-view features to improve feature expression capability;

[0013] S5, prompt mixing: fuse cross-view features and optimal prompts to generate final prompt embedding;

[0014] S6, tampering detection and positioning: input the fusion prompt and the feature into the mask decoder of the visual base model, and output the tampering area positioning result.

[0015] Further, the multiple complementary views in step S1 include:

[0016] RGB view, spatial residual map (SRM) view, Bayer noise view and noise fingerprint (Noiseprint) view;

[0017] Each view processes the image through a dedicated feature extractor and a segmenter to generate corresponding view features.

[0018] Further, step S2 specifically includes:

[0019] Probability distribution prediction is performed on each view feature to generate a tampering confidence map;

[0020] Weighted average of each view confidence map to generate an integrated distribution;

[0021] The optimal prompt distribution is selected by minimizing the focus loss of each view and integrated distribution and real label.

[0022] Further, the step S3 specifically comprises:

[0023] The sum of segmentation loss of each view confidence map and optimal prompt distribution is calculated as cross-view consistency loss, and the inter-view prompt consistency is optimized. Figure One The inter-view prompt consistency is optimized.

[0024] Further, the step S4 specifically comprises:

[0025] The multi-view features are aligned by bicubic interpolation;

[0026] The 1x1 convolution is adopted to enhance the feature expression, and the visual basic model coding feature is spliced;

[0027] After the features are integrated by the convolution block, the global average pooling and the full connection layer are used for semantic selection attention weighting to generate the fusion features.

[0028] Further, the step S5 specifically comprises:

[0029] The output features of the cross-view feature perception module and the optimal prompt coding are fused by the multi-layer perception to generate the final prompt embedding.

[0030] Further, the model is trained by jointly optimizing the following loss functions in the training stage:

[0031] The mask decoder segmentation loss, the multi-view segmentation loss, the cross-view consistency loss and the image-level prediction loss; Figure One

[0032] The image-level prediction loss is calculated by the binary cross entropy to calculate the difference between the predicted label and the real label.

[0033] Further, the integrated distribution is used as the prompt input in the inference stage, and the end-to-end tampering positioning is realized by the cross-view feature fusion and the optimal prompt selection.

[0034] Further, the visual basic model is a segmentation basic model (SAM), and the mask decoder outputs the pixel-level tampering positioning result according to the fusion prompt and the feature.

[0035] A computer program product comprises a computer program, which, when executed by a processor, implements the image tampering detection method based on the visual basic model.

[0036] The present application has the following beneficial effects:

[0037] ​The application provides an image tampering detection method IMDPrompter based on a visual basic model. Figure One The cross-view consistency optimization and feature perception fusion enhance the robustness of the model, effectively overcome the performance degradation problem under the interference of image compression, blurring and the like, and finally realize high-precision positioning of a complex tampering scene, and comprehensively improve the accuracy and adaptability of image forensics.

[0038] Compared with the prior art, the application has the following advantages:

[0039] Cross-view automatic prompt generation: IMDPrompter generates prompt information automatically and combines multiple views (such as RGB, SRM, Bayer and Noiseprint views), overcomes the dependence of existing methods on manual prompts, and improves the automation and efficiency of detection.

[0040] Enhanced cross-dataset generalization capability: by fusing multi-view information, IMDPrompter improves the cross-dataset generalization capability of image tampering detection, and especially shows stronger robustness in different datasets and environments.

[0041] High-precision tampering positioning capability: the application significantly improves the detection and positioning accuracy of image tampering by cross-view prompt consistency optimization and optimal prompt selection, and especially in complex image tampering cases, can accurately identify and locate the tampered region.

[0042] Other beneficial effects in the embodiments of the application will be further described below. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is the overall flowchart of the image tampering detection method based on the visual basic model of the application.

[0044] Figure 2 is the algorithm overall framework diagram of the image tampering detection method IMDPrompter of the embodiment of the application.

[0045] Figure 3 is the cross-view feature perception module cross-view feature perception unit structure diagram of the embodiment of the application.

[0046] Figure 4is a visual comparison diagram of image tampering detection and positioning results of an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following detailed description of the embodiments of the present application. It should be emphasized that the following description is merely exemplary, but not intended to limit the scope of the present application and its applications.

[0048] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include one or more features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0049] The present application provides an image tampering detection method IMDPrompter based on a visual basic model. The visual basic model SAM is first applied to the image tampering detection task. By automatically generating prompt information and combining multiple view information, the performance of image tampering detection is effectively improved. The present application eliminates the dependence of manual prompts, and enhances the model's cross-dataset generalization ability and robustness through cross-view feature fusion and optimal prompt selection.

[0050] Referring to Figure 1 and Figure 2 , the present application provides an image tampering detection method based on a visual basic model, comprising the following steps:

[0051] Step S1, cross-view automatic prompt generation: image features are extracted from multiple complementary views, and unified prompt information is generated by fusion, eliminating the dependence of manual prompts.

[0052] In some embodiments, the multiple complementary views specifically include: RGB view, spatial residual map (SRM) view, Bayer noise view and noise fingerprint (Noiseprint) view; each view processes the image through a dedicated feature extractor and a segmenter, respectively, to generate corresponding view features.

[0053] Step S2, optimal prompt selection: based on minimizing segmentation loss, the optimal prompt is selected from the prompts generated from multiple views.

[0054] In some embodiments, step S2 specifically includes: performing probability distribution prediction on each view feature to generate a tampering confidence map; weighted average of each view confidence map to generate an integrated distribution; by minimizing the focus loss of each view and the integrated distribution and the real label, the optimal prompt distribution is selected.

[0055] Step S3, cross-view Figure OneConsistency loss: enhance the consistency of different view hints through consistency loss.

[0056] In some embodiments, step S3 specifically includes: calculating the sum of the segmentation loss of each view confidence map and the optimal hint distribution as the cross-view Figure One Consistency loss: optimize the consistency of inter-view hints.

[0057] Step S4, cross-view feature perception fusion: align and fuse multi-view features to improve feature expression capability.

[0058] In some embodiments, referring to Figure 3 , step S4 specifically includes: aligning multi-view features through bicubic interpolation; using 1x1 convolution to enhance feature expression and splicing with visual base model encoded features; after integrating features through convolution blocks, performing semantic selection attention weighting through global average pooling and fully connected layers to generate fused features.

[0059] Step S5, hint mixing: fuse cross-view features and optimal hints to generate final hint embedding.

[0060] In some embodiments, step S5 specifically includes: fusing the output features of the cross-view feature perception module and the optimal hint encoding through a multi-layer perception machine to generate the final hint embedding.

[0061] Step S6, tampering detection and positioning: input the fused hints and features into the mask decoder of the visual base model to output tampering area positioning results.

[0062] In some embodiments, the training phase trains the model by jointly optimizing the following loss functions: mask decoder segmentation loss, multi-view segmentation loss, cross- Figure One Consistency loss, image-level prediction loss; wherein the image-level prediction loss calculates the difference between the predicted label and the true label through binary cross-entropy.

[0063] In some embodiments, the inference phase uses an integrated distribution as a hint input to realize end-to-end tampering positioning through cross-view feature fusion and optimal hint selection.

[0064] In some embodiments, the visual base model is a segmentation base model (SAM), and the mask decoder outputs pixel-level tampering positioning results according to the fused hints and features.

[0065] The specific embodiments of the present application, algorithm examples and experimental verification are further described below.

[0066] An image tampering detection method IMDPrompter based on a visual base model, the implementation process mainly includes image input, feature extraction, automatic hint generation, optimal hint selection, cross-view feature fusion, cross-viewFigure One Consistency optimization, cue fusion, and image segmentation. For example... Figure 2 As shown, the algorithm architecture of IMDPrompter mainly includes:

[0067] Cross-view automated prompt generation: IMDPrompter eliminates the reliance on manual prompt input by automatically generating prompts across different views. The automated prompt generation framework extracts image features from different views (RGB view, SRM view, Bayer view, and Noiseprint view) and merges them into a valid prompt for SAM to perform image tampering detection.

[0068] Optimal Cue Selection (OPS): During cue generation, IMDPrompter uses the Optimal Cue Selection (OPS) module to select the optimal cue from multiple views, minimizing the segmentation loss. The OPS module optimizes cue information from different views, improving image segmentation accuracy.

[0069] Cross-view Figure One Consistency Module (CPC): To ensure the consistency of prompts from different views, this invention proposes Cross-View Prompt Consistency Enhancement Loss (CPC), which enhances the consistency of prompts between views, thereby improving detection accuracy.

[0070] Cross-view Feature Awareness Module (CFP): Through the Cross-view Feature Awareness Module (CFP), IMDPrompter integrates feature information from RGB, SRM, Bayer, and Noiseprint views, further improving SAM's feature representation capability in the image segmentation process and enhancing the effect of image tamper detection.

[0071] The Prompt Hybrid Module (PMM) fuses features from multiple views and the optimal prompt using a multilayer perceptron (MLP) to generate the final prompt information for image tamper detection and localization using the SAM model.

[0072] The following details the specific implementation process of IMDPrompter.

[0073] Step 1: Image Input and Feature Extraction

[0074] The image I to be detected enters the IMDPrompter system through the input terminal. The image is first processed into four views: RGB view Φ RGB (I) SRM view Φ SRM (I) Bayer View Φ Bayer (I) and Noiseprint view Φ Noiseprint(I). Each view is responsible for extracting different types of features and sending them to the corresponding feature segmenter for processing.

[0075] SAM encoder: Input image I directly into SAM image encoder Φ sam-enc Obtain the encoded feature F sam .

[0076] RGB View: The RGB view provides standard color image information, including the image's basic color and brightness characteristics. This is achieved through the RGB image segmenter Φ. Seg-RGB Extract feature information to obtain RGB feature f RGB .

[0077] SRM View: The SRM (Spatial Residual Map) view extracts residual features from an image, used to detect anomalous regions. This is achieved through the SRM noise map extractor Φ. SRM Features were extracted and processed using the SRM view segmenter Φ Seg-SRM Segmentation is performed to obtain SRM features f SRM .

[0078] Bayer View: The Bayer view primarily captures noise information in an image, especially in digital image processing. The Bayer filter can effectively identify image artifacts and compression errors. (The Bayer noise map Φ...) Bayer The extractor extracts features and passes them through the Bayer view segmenter Φ. Seg-Bayer Segmentation is performed to obtain Bayer features f Bayer .

[0079] Noiseprint View: The Noiseprint view is used to extract the noise fingerprint of an image, capturing potential traces of tampering. Through the Noiseprint noise map Φ Noiseprint The extractor extracts features and passes them through the Noiseprint view segmenter Φ. Seg-Noiseprint Segmentation is performed to obtain the Noiseprint feature f Noiseprint .

[0080] F sam =Φ sam-enc (I)

[0081] f RGB =Φ Seg-RGB (I)

[0082] f SRM =Φ Seg-SRM (Φ SRM (I))

[0083] f Bayer =Φ Seg-Bayer (ΦBayer (I))

[0084] f Noiseprint =Φ Seg-Noiseprint (Φ Noiseprint (I))

[0085] Step 2: Automated suggestion generation and optimal suggestion selection (OPS)

[0086] In IMDPrompter, automatically generated prompts guide SAM in image segmentation and tamper detection. This process is achieved through the following steps:

[0087] Hint generation: Based on four view features f RGB f SRM f Bayer f Noiseprint Through the corresponding classifier Φ RGB-CLS Φ SRM-CLS Φ Bayer-CLS Φ Noiseprint-CLS Generate the corresponding mask probability distribution P RGB P SRM P Bayer P Noiseprint These probability distributions represent the confidence level that each pixel belongs to the tampered region.

[0088] P RGB =Φ RGB-CLS (f RGB )

[0089] P SRM =Φ SRM-CLS (f SRM )

[0090] P Bayer =Φ Bayer-CLS (f Bayer )

[0091] P Noiseprint =Φ Noiseprint-CLS (f Noiseprint )

[0092] Ensemble distribution: The probability distributions generated from the four view features are weighted and averaged to form an ensemble distribution P. Ens This is the combined result of all view suggestions.

[0093]

[0094] Optimal suggestion: Calculate P separately RGB P SRM P Bayer P Noiseprint P EnsThe focusing loss function L with the true label G Seg The optimal mask probability distribution P is selected by minimizing the segmentation loss function. Opt This improves the accuracy of segmentation.

[0095]

[0096] Hint encoding: using the optimal mask probability distribution P Opt Derive the optimal hint M Opt And bounding box hint B Opt P Opt and M Opt Input to SAM prompt encoder Φ pro-enc In the process, the final prompt code F is generated. Opt It is used for image segmentation.

[0097] F Opt =Φ pro-enc (M Opt B Opt )

[0098] Step 3: Cross-view Figure One Consistency optimization (CPC)

[0099] To ensure consistency of tooltip information across different views, this invention designs a cross-view tooltip consistency module (CPC). This module enhances tooltip consistency across different views in the following ways:

[0100] Consistency Loss Function: Cross-view Hint Consistency Enhancement Loss L CPC It was used to optimize the tooltips generated for different views, ensuring they matched the optimal tooltip P. Opt Consistency between views. Specifically, this means reducing differences in the prompt generation process between different views, thereby improving overall detection accuracy.

[0101] L CPC =L Seg (P RGB ,P Opt )+L Seg (P SRM ,P Opt )+L Seg (P Bayer ,P Opt )+L Seg (P Noiseprint ,P Opt )

[0102] Step 4: Cross-view Feature Fusion (CFP)

[0103] After feature extraction, the feature information of each view enters the cross-view feature perception module (e.g.,Figure 3 (As shown). The goal of this module is to enhance the expressive power of images by fusing features from different views, providing richer information for subsequent segmentation and tamper detection. The specific steps are as follows:

[0104] Feature alignment: f extracted via bicubic interpolation RGB f SRM f Bayer f Noiseprint View features are aligned to ensure spatial consistency.

[0105] Feature concatenation: The alignment features of the four views are processed through 1×1 convolution to further enhance the expressive power of the features. This is then combined with the image encoding features F. sam They are spliced ​​together to form a more representative feature vector.

[0106] Feature fusion: The concatenated features are fed into two convolutional blocks (3×3 convolution, instance normalization, and GELU activation) to further integrate and optimize these features.

[0107] Semantic Selective Attention: Semantic selective attention is implemented through global average pooling, fully connected layers, and GELU activation, enabling the model to focus more on key features, thereby generating fused cross-view features F. CFP .

[0108] Step 5: Hint Fusion (PMM) and Image Tampering Detection

[0109] Finally, the prompts from multiple views are fused together by the prompt fusion module. This module's function is to fuse prompts from the Cross-View Feature Awareness (CFP) module and the Optimal Prompt Selection (OPS) module to generate the final prompt.

[0110] Hint fusion: through multilayer perceptron Φ MLP Feature information from the CFP and OPS modules is fused. This process integrates cue information from different sources to generate a unified cue embedding F. mix .

[0111] F mix =Φ MLP ([F CFP ,F Opt ])

[0112] Image tampering location: Fusion prompt information F mix and cross-view feature F CFP The mask decoder Φ is input into SAM sam-dec In the process, SAM performs image segmentation based on the prompt information and calculates the mask prediction probability P. samThe location of the tampered area is then identified. The image segmentation results clearly show the tampered parts in the image, providing accurate data support for subsequent evidence collection or analysis.

[0113] P sam =Φ sam-dec (F CFP ,F mix )

[0114] Step 6: Training and Reasoning Process

[0115] Training process: During training, IMDPrompter uses a multi-view segmentation loss L... Seg-pro , indicating consistency loss L CPC Segmentation loss L of SAM decoder Seg-sam and image-level prediction loss L img-pre Joint optimization is performed. Among them, the image-level prediction loss L... img-pre Image-level prediction based on image-level label y Binary cross-entropy loss function L BCE The model was trained using the AdamW optimizer with appropriate learning rate scheduling and batch size settings to ensure convergence and robustness.

[0116] L = L Seg-sam +λ1L Seg-pro +λ2L CPC +λ3L img-pre

[0117] L Seg-sam =L Seg (P sam ,G)

[0118] L Seg-pro =L Seg (P RGB ,G)+L Seg (P SRM ,G)+L Seg (P Bayer ,G)+L Seg (P Noiseprint ,G)

[0119]

[0120] Reasoning process: During the reasoning phase, IMDPrompter utilizes the optimal cue selection strategy learned during training, defaulting to P. Ens As input prompts, image tampering detection and localization are performed. Since there are no truth labels, the inference process relies on optimal prompt generation and the fusion of cross-view information.

[0121] Experimental results

[0122] Table 1 is a performance evaluation table for pixel-level tampering detection provided by this invention. As can be seen from Table 1, the method of this invention achieves optimal performance under both optimal and fixed threshold conditions. Furthermore, compared to the baseline method FCN, the method of this invention achieves a performance improvement of over 10% in most metrics.

[0123] Table 2 is a performance evaluation table for image-level tampering detection provided by this invention. As can be seen from Table 2, the method of this invention achieves the best performance on both the in-domain test set CASIA and the out-of-domain test sets Cover, Columbia, and IMD. Furthermore, it can be seen that compared to the baseline method FCN, the method of this invention achieves significant performance improvements in most metrics.

[0124] Table 1

[0125]

[0126] Table 2

[0127]

[0128] Figure 4 This is a visual comparison diagram of the image tampering detection and localization results provided by the present invention. Figure 4 As can be seen, ManTra-Net, MVSS-Net, and Trufor all struggle to obtain a complete response to the tampered region and fail to achieve refined boundary predictions, while the method of this invention achieves accurate image tampering detection results, further demonstrating the superiority of the method of this invention.

[0129] Experimental results show that the IMDPrompter of this invention eliminates the reliance on manual prompts through cross-view feature perception and automated prompt generation mechanisms. Furthermore, through innovative designs such as cross-view information fusion and optimal prompt selection, it not only effectively improves the performance of image tampering detection but also enhances the model's generalization ability and robustness.

[0130] In summary, this invention provides an image tampering detection method based on a visual fundamental model. Compared with existing technologies, the main innovative advantages of this invention include:

[0131] Cross-view automated tooltip generation: IMDPrompter overcomes the dependence on manual tooltips in existing methods by automatically generating tooltip information and combining multiple views (such as RGB, SRM, Bayer, and Noiseprint views), thereby improving the automation and efficiency of detection.

[0132] Enhanced cross-dataset generalization: By fusing multi-view information, IMDPrompter improves the cross-dataset generalization of image tampering detection, especially showing stronger robustness in different datasets and environments.

[0133] High-precision tamper location capability: This invention significantly improves the detection and location accuracy of image tampering through cross-view prompt consistency optimization and optimal prompt selection, especially in complex image tampering cases, it can accurately identify and locate the tampered area.

[0134] This invention also provides a storage medium for storing a computer program, which, when executed, performs at least the methods described above.

[0135] This invention also provides a control device, including a processor and a storage medium for storing a computer program; wherein the processor executes the computer program by performing at least the method described above.

[0136] This invention also provides a processor that executes a computer program, at least performing the methods described above.

[0137] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc or CD-ROM; magnetic surface memory can be disk storage or magnetic tape storage. The storage media described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0138] In the several embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0139] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0140] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0141] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0143] The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0144] The features disclosed in the several product embodiments provided by this invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0145] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0146] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or application, should be considered within the scope of protection of the present invention.

Claims

1. A method for detecting image tampering based on a visual fundamental model, characterized in that, Includes the following steps: S1. Automated cross-view prompt generation: Extracts image features from multiple complementary views and merges them to generate unified prompt information, eliminating reliance on manual prompts; S2. Optimal prompt selection: Based on minimizing the segmentation loss, select the optimal prompt from the prompts generated by the multi-view; S3, Cross-view Consistency Optimization: Enhance the consistency of prompts across different views by using consistency loss; S4. Cross-view feature perception fusion: Align and fuse features from multiple views to improve feature representation capabilities; S5, Hint Blending: Integrates cross-view features with the optimal hints to generate the final hint embedding; S6. Tampering Detection and Localization: The mask decoder that integrates the fusion of prompts and features into the visual base model outputs the tampering region localization result.

2. The method as described in claim 1, characterized in that, The multiple complementary views mentioned in step S1 include: RGB view, Spatial Residual Map (SRM) view, Bayer noise view, and Noiseprint view; Each view is processed by a dedicated feature extractor and segmenter to generate corresponding view features.

3. The method as described in claim 1 or 2, characterized in that, Step S2 specifically includes: Probability distribution prediction is performed on the features of each view to generate a tampering confidence map; The weighted average confidence plots of each view are used to generate an integrated distribution. The optimal cue distribution is selected by minimizing the focus loss between each view and the integrated distribution and the actual label.

4. The method as described in claim 3, characterized in that, Step S3 specifically includes: Calculate the sum of the segmentation losses of the confidence maps of each view and the optimal cue distribution, and use it as the cross-view consistency loss to optimize the cue consistency between views.

5. The method as described in claim 1 or 2, characterized in that, Step S4 specifically includes: Align multi-view features using bicubic interpolation; 1×1 convolution is used to enhance feature representation, and it is concatenated with the features encoded by the visual base model; After integrating features through convolutional blocks, semantic selection attention weighting is performed through global average pooling and fully connected layers to generate fused features.

6. The method as described in claim 1 or 2, characterized in that, Step S5 specifically includes: The final prompt embedding is generated by fusing the output features of the cross-view feature perception module with the optimal prompt encoding through a multilayer perceptron.

7. The method as described in claim 1 or 2, characterized in that, During the training phase, the model is trained by jointly optimizing the following loss functions: Mask decoder segmentation loss, multi-view segmentation loss, cross-view consistency loss, image-level prediction loss; The image-level prediction loss is calculated by using binary cross-entropy to determine the difference between the predicted label and the true label.

8. The method as described in claim 1 or 2, characterized in that, The reasoning phase uses integrated distribution as the prompt input, and achieves end-to-end tamper location through cross-view feature fusion and optimal prompt selection.

9. The method as described in claim 1 or 2, characterized in that, The visual base model is a segmentation base model (SAM), whose mask decoder outputs pixel-level tampering localization results based on fusion cues and features.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image tampering detection method based on the visual fundamental model as described in any one of claims 1 to 9.

Images