Image tampering identification method and device, terminal, electronic equipment and storage medium

Abstract

The invention relates to an image tampering recognition method and device, a terminal, electronic equipment and a storage medium. The method comprises the steps that a target image to be subjected to tampering recognition is acquired; performing tampering trace detection based on an expert model on the target image to obtain a trace detection report; retrieving at least one reference image similar to the target image, and comparing the target image with the at least one reference image to obtain a comparison report; and through a large-scale generative language model, performing fusion decision on the trace detection report and the comparison report to obtain a tampering identification result corresponding to the target image. According to the invention, image tampering identification can be accurately carried out.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an image tampering identification method, apparatus, terminal, electronic device, and storage medium. Background Technology

[0002] With the development of image editing and AI-generated image technologies, the threshold for tampering with digital images has been significantly lowered, while forgery methods have become increasingly sophisticated. Traditional manual review methods are not only inefficient when processing massive amounts of electronic documents, but also largely ineffective against sophisticated tampering that is not easily detectable to the naked eye. AI-based solutions also suffer from poor generalization, only working in specific scenarios, leading to a significant drop in accuracy when facing new tampering techniques. Summary of the Invention

[0003] This disclosure provides an image tampering identification method, apparatus, terminal, electronic device, and storage medium to solve problems in related technologies. The technical solution of this disclosure is as follows: According to a first aspect of the present disclosure, an image tampering identification method is provided, the method comprising: Acquire the target image to be identified as tampered with; The target image is subjected to tampering trace detection based on an expert model, and a trace detection report is obtained; Retrieve at least one reference image that is similar to the target image, and obtain a comparison report by comparing the target image with the at least one reference image; A large-scale generative language model is used to fuse the trace detection report and the comparison report to obtain the tampering identification result corresponding to the target image.

[0004] In one exemplary implementation, the method includes: Perform attribute data consistency detection on the target image to obtain a consistency detection report; The step of fusing the trace detection report and the comparison report using a large-scale generative language model to obtain the tampering identification result corresponding to the target image includes: The large-scale generative language model is used to fuse the trace detection report, the comparison report, and the consistency detection report to obtain the tampering identification result.

[0005] In one exemplary implementation, the step of performing tampering trace detection on the target image based on an expert model to obtain a trace detection report includes: Perform at least one preprocessing operation on the target image to obtain the preprocessing result corresponding to each preprocessing operation; Using the expert model corresponding to the target image, the fake region is identified in each of the preprocessing results to obtain an image marked with the identification result; Textual description information is obtained based on the differences between the target image and the image labeled with the recognition results, using natural language description. The first inference model is used to infer forgery traces in the target image based on the text description information to obtain the trace detection report. The first inference model belongs to a large-scale generative language model.

[0006] In one exemplary implementation, the trace detection report includes at least one of the following: Confidence level of forgery traces; Explanation of the reasoning process behind the forgery traces; If the forgery is confirmed, the proposed impact on the target business associated with the target image; If the confidence level of the forgery traces does not meet the preset confidence level requirement, other verification methods are proposed to determine whether the target image has been tampered with.

[0007] In one exemplary implementation, before retrieving at least one reference image similar to the target image and obtaining a comparison report by comparing the target image with the at least one reference image, the method includes: A multimodal large-scale generative language model is used to extract features from the target image to obtain the corresponding features to be compared. Based on the feature to be compared, a search is performed in the historical feature database to obtain at least one reference feature similar to the feature to be compared. The historical feature database is used to record features corresponding to historical images. The image corresponding to each of the aforementioned reference features is used as the corresponding reference image.

[0008] In one exemplary implementation, retrieving at least one reference image similar to the target image, and obtaining a comparison report by comparing the target image with the at least one reference image, includes: Based on the second inference model, the target image is compared with each of the reference images to obtain the comparison report. The second inference model belongs to a multimodal large-scale generative language model.

[0009] In one exemplary implementation, the step of performing attribute data consistency detection on the target image to obtain a consistency detection report includes: Extract the editing information of the software corresponding to the target image, wherein the editing information includes at least one of creation information and modification information; Extract the timestamp information corresponding to the target image; Verify the consistency between the edited information and the timestamp information to obtain the consistency detection report.

[0010] In one exemplary implementation, the preprocessing operation on the target image includes performing at least one of the following on the target image: Compression error accumulation analysis, wavelet transform, spectrum analysis, multi-scale analysis, histogram equalization, and adaptive noise filtering.

[0011] In one exemplary implementation, the first inference model is a multimodal large-scale generative language model. The step of using the first inference model to infer forgery traces from the target image based on the text description information to obtain the trace detection report includes: Using the first inference model, based on the source image and the text description information, forgery trace inference is performed on the target image to obtain the trace detection report. The source image includes at least one of the following: the target image, an associated image obtained by preprocessing the target image, and the image marked with the recognition result.

[0012] According to a second aspect of the present disclosure, an image tampering identification apparatus is provided, the apparatus comprising: The image acquisition module is configured to acquire the target image to be identified as being tampered with; The trace detection module is configured to perform tamper trace detection on the target image based on an expert model and obtain a trace detection report; The comparison and retrieval module is configured to perform a search for at least one reference image that is similar to the target image, and to obtain a comparison report by comparing the target image with the at least one reference image; The fusion decision module is configured to perform a fusion decision on the trace detection report and the comparison report using a large generative language model to obtain the tampering identification result corresponding to the target image.

[0013] In one exemplary embodiment, the apparatus further includes: The consistency detection module is configured to perform attribute data consistency detection on the target image and obtain a consistency detection report; The fusion decision module is configured to execute: The large-scale generative language model is used to fuse the trace detection report, the comparison report, and the consistency detection report to obtain the tampering identification result.

[0014] In one exemplary implementation, the trace detection module is configured to perform: Perform at least one preprocessing operation on the target image to obtain the preprocessing result corresponding to each preprocessing operation; Using the expert model corresponding to the target image, the fake region is identified in each of the preprocessing results to obtain an image marked with the identification result; Textual description information is obtained based on the differences between the target image and the image labeled with the recognition results, using natural language description. The first inference model is used to infer forgery traces in the target image based on the text description information to obtain the trace detection report. The first inference model belongs to a large-scale generative language model.

[0015] In one exemplary implementation, the trace detection report includes at least one of the following: Confidence level of forgery traces; Explanation of the reasoning process behind the forgery traces; If the forgery is confirmed, the proposed impact on the target business associated with the target image; If the confidence level of the forgery traces does not meet the preset confidence level requirement, other verification methods are proposed to determine whether the target image has been tampered with.

[0016] In one exemplary implementation, the comparison and retrieval module is configured to perform: A multimodal large-scale generative language model is used to extract features from the target image to obtain the corresponding features to be compared. Based on the feature to be compared, a search is performed in the historical feature database to obtain at least one reference feature similar to the feature to be compared. The historical feature database is used to record features corresponding to historical images. The image corresponding to each of the aforementioned reference features is used as the corresponding reference image.

[0017] In one exemplary implementation, the comparison and retrieval module is configured to perform: Based on the second inference model, the target image is compared with each of the reference images to obtain the comparison report. The second inference model belongs to a multimodal large-scale generative language model.

[0018] In one exemplary implementation, the consistency detection module is configured to perform: Extract the editing information of the software corresponding to the target image, wherein the editing information includes at least one of creation information and modification information; Extract the timestamp information corresponding to the target image; Verify the consistency between the edited information and the timestamp information to obtain the consistency detection report.

[0019] In one exemplary implementation, the preprocessing operation on the target image includes performing at least one of the following on the target image: Compression error accumulation analysis, wavelet transform, spectrum analysis, multi-scale analysis, histogram equalization, and adaptive noise filtering.

[0020] In one exemplary implementation, the first inference model is a multimodal large-scale generative language model, and the trace detection module is configured to execute: Using the first inference model, based on the source image and the text description information, forgery trace inference is performed on the target image to obtain the trace detection report. The source image includes at least one of the following: the target image, an associated image obtained by preprocessing the target image, and the image marked with the recognition result.

[0021] According to a third aspect of the present disclosure, an electronic device is provided, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the image tampering identification method as described in any of the above embodiments.

[0022] According to a fourth aspect of the present disclosure, a computer storage medium is provided, wherein when instructions in the computer storage medium are executed by a processor of an electronic device, the electronic device performs the image tampering identification method described in any of the above embodiments.

[0023] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including a computer program that, when executed by a processor, implements the image tampering recognition method described in any of the above embodiments.

[0024] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: The image tampering recognition method provided in this disclosure integrates tampering trace detection by an expert model to output a trace detection report and an image comparison report. A large-scale generative language model is then used to comprehensively fuse these reports to obtain the tampering recognition result. Compared to related technologies that rely solely on a single model for tampering recognition, this disclosure offers a multi-path collaborative approach. A significant feature of this approach is the combination of pre-processing with an expert model and post-processing with a large-scale generative language model, thus addressing the poor generalization of single models. This multi-method approach enhances the accuracy and adaptability of tampering recognition. In practical applications, this multi-path collaborative approach can effectively handle complex and diverse image tampering methods. Pre-processing with an expert model accurately captures potential tampering traces in the image, while the large-scale generative language model further performs in-depth analysis and logical reasoning on the detection results, compensating for potential misjudgments or omissions that may occur with single models in specific scenarios. Furthermore, this scheme possesses strong scalability, allowing for the flexible introduction of more advanced detection technologies and algorithms to adapt to constantly evolving image tampering methods. This multi-layered, multi-dimensional comprehensive decision-making mechanism enables the entire system to maintain high robustness and reliability even when facing various unknown tampering techniques.

[0025] This disclosure addresses the technical limitations of single-model image tampering detection. Specifically, relying solely on large-scale generative language models or expert models for tampering detection leads to decreased recognition rates when faced with small samples or data distributions varying significantly across different domains, resulting in inconsistent performance. Furthermore, traditional deep learning models heavily depend on high-quality, large-scale labeled data, and their "black box" nature makes the results lack interpretability, hindering the complete trust of reviewers. This disclosure proposes a fusion and complementary approach, allowing different models to collaborate in their strongest areas. This not only solves the problems of decreased recognition rates and limited application scenarios but also enhances interpretability, enabling reviewers to more clearly understand the model's decision-making process and thus increasing trust in the recognition results. By organically combining the advantages of expert models and large-scale generative language models, higher adaptability can be demonstrated in complex scenarios. For example, when dealing with subtle traces of image tampering, expert models can provide precise local feature analysis, while generative language models can further verify and supplement the expert model's results through logical deduction based on an understanding of global information. This collaborative mechanism not only improves identification accuracy but also significantly reduces false alarm and false negative rates. Furthermore, the technical solution is designed to fully consider the diversity and dynamic changes required in practical applications, ensuring rapid iteration and upgrades as new tampering methods emerge, and maintaining a consistently high level of detection capability.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this disclosure, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0028] Figure 1 This is a flowchart illustrating an image tampering identification method according to an exemplary embodiment; Figure 2 This is a flowchart illustrating a tamper trace detection scheme according to an exemplary embodiment; Figure 3 This is a schematic diagram illustrating the reasoning process of forgery traces according to an exemplary embodiment; Figure 4 This is a schematic diagram of a reference image determination method according to an exemplary embodiment; Figure 5 This is a schematic diagram illustrating an implementation process for determining a reference image according to an exemplary embodiment; Figure 6This is a schematic diagram illustrating the process of image tampering identification according to an exemplary embodiment; Figure 7 This is a block diagram of an image tampering identification device according to an exemplary embodiment; Figure 8 This is a structural block diagram of a computer device according to an exemplary embodiment. Figure 1 ; Figure 9 This is a structural block diagram of a computer device according to an exemplary embodiment. Figure 2 . Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0030] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0031] Figure 1 This is a flowchart illustrating an image tampering identification method according to an exemplary embodiment. The image tampering identification method can be applied to an electronic device, which can be implemented by a server or a terminal alone, or by a terminal and a server working together. The terminal can be, but is not limited to, physical devices such as smartphones, tablets, laptops, desktop computers, smart speakers, smart wearable devices, digital assistants, augmented reality devices, and virtual reality devices, and can also include software such as applications running on the physical device. The server can be, but is not limited to, a standalone server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms, etc. (Refer to...) Figure 1 As shown, the method includes the following steps.

[0032] In S110, the target image to be identified as tampered with is acquired.

[0033] This disclosure does not limit the target image. For example, the target image can be any image file containing visual information, such as photographs, medical images, satellite images, scanned text, etc. These images can originate from different devices or systems, including but not limited to digital cameras, scanners, medical imaging equipment, or remote sensing equipment. The format of the target image can also be diverse, including JPEG, PNG, BMP, or other common image storage formats. Regardless of its source or format, as long as the image may show signs of tampering, it can be used as input for analysis and identification in this method.

[0034] In some scenarios, with the comprehensive digitalization of enterprise operations, critical documents such as financial invoices, business contracts, and qualification certificates are increasingly being circulated and archived in electronic image form. Simultaneously, the widespread adoption of image editing software (such as Adobe Photoshop) and artificial intelligence (AI) generation technologies has brought significant financial and compliance risks to enterprises due to digital image tampering. Image tampering methods are becoming increasingly numerous and sophisticated, rendering single deep learning models ineffective in providing effective tampering detection results. For the growing number of tampering techniques, single models lack generalization ability, proving effective only in specific tampering domains and failing to identify tampering in many other situations. For example, they might identify a tampered image as authentic, or identify tampering but incorrectly pinpoint the location of the tampering.

[0035] To address the issues of low accuracy and limited application scenarios of single-model image tampering detection, this disclosure proposes a multi-model collaborative tampering detection approach. This approach is characterized by high scenario adaptability, so step S110 does not limit the scenario or the source of the target image in the scenario. Any image that needs to be tampered with in any scenario can be used as the target image in S110.

[0036] S120. Perform tampering trace detection on the target image based on an expert model to obtain a trace detection report.

[0037] This disclosure does not limit the scope of expert models; they can be understood as models selected for tamper detection based on the scene or characteristics of the target image. For example, an expert model can be a deep learning-based image analysis model or a hybrid model combining traditional image processing techniques with machine learning algorithms. The selection criteria include, but are not limited to, the target image's resolution, color distribution, texture features, basic content, and possible types of tampering. For instance, when detecting images with complex backgrounds, models skilled in segmentation and feature extraction can be chosen; while for high-resolution images, models with high-precision analysis capabilities are preferred.

[0038] For example, if the target image contains key documents such as financial invoices, business contracts, and qualification certificates from a company's comprehensive digital operations, the expert model may prefer to select a model that is sensitive to text areas and can identify subtle signs of tampering. Such models are better able to capture potential modifications to areas such as text, seals, and signatures. Furthermore, considering that documents such as financial invoices often have fixed formats and standards, the expert model can also combine rule-based matching to further improve detection accuracy. For images involving multiple types of information, such as business contracts or qualification certificates, the expert model can use algorithms with comprehensive analytical capabilities to address various potential tampering methods, such as partial replacement, content erasure, or the addition of forged information.

[0039] S130. Retrieve at least one reference image similar to the target image, and obtain a comparison report by comparing the target image with the at least one reference image.

[0040] In step S120, the tampering trace detection process performed by the expert model is essentially a single-image blind detection. This process does not utilize historical images and tends to search for subtle signs of tampering within the target image itself, thereby generating a trace detection report. As a supplement to step S120, step S130 introduces a reference image comparison mechanism. This method effectively compensates for the shortcomings of single-image blind detection. By retrieving reference images similar to the target image, a more comprehensive comparison benchmark can be established, thereby discovering subtle differences in the target image that may have been overlooked. For example, when certain areas in the target image are inconsistent with the reference image, these areas are likely to become points of suspicion for tampering. Furthermore, the generation of the comparison report not only relies on pixel-level difference analysis but can also incorporate semantic information for deeper judgment, further improving the reliability of the detection results. For example, in the process of generating the comparison report, the introduction of semantic information can help identify potential connections between the target image and the reference image. For instance, by analyzing object categories, scene layouts, and contextual relationships in the images, it is possible to more accurately determine whether certain differences originate from tampering. This method is particularly suitable for image analysis in complex scenarios and can effectively reduce the false alarm rate. This step, by comprehensively utilizing the target image's own features and external reference data, provides a more comprehensive solution for tamper detection.

[0041] S140. Using a large-scale generative language model, a fusion decision is made between the trace detection report and the comparison report to obtain the tampering identification result corresponding to the target image.

[0042] The operations performed in steps S120 and S130 can be implemented by artificial intelligence models, such as deep learning models. These models are black-box models, and their output lacks interpretability. This can lead to distrust of the output by the auditors who identify image tampering. In order to improve trust, this disclosure uses a large generative language model to perform a fusion decision on the trace detection report and the comparison report to obtain the tampering identification result corresponding to the target image.

[0043] Large-scale generative language models (GLGMs) are deep learning models capable of processing and generating natural language, characterized by a large number of parameters and powerful language understanding and generation capabilities. These models, pre-trained on massive amounts of text data, can capture complex language patterns and semantic information, thus performing exceptionally well in various tasks. In image tampering detection scenarios, GLGMs can integrate key information from trace detection and comparison reports, and combine this with contextual semantics to generate more comprehensive and credible decision results. This approach not only improves the accuracy of the recognition results but also enhances the interpretability of the output, providing auditors with more intuitive judgment criteria. In this way, the model can maintain high efficiency while further meeting the demands for transparency and credibility in practical applications.

[0044] In summary, the image tampering recognition method disclosed herein integrates expert model tampering trace detection to output a trace detection report and image comparison report. A large-scale generative language model is then used to comprehensively fuse these reports, resulting in a tampering recognition result. Compared to related technologies that rely solely on a single model for tampering recognition, this disclosure provides a multi-path collaborative approach. A significant feature of this approach is the combination of expert model pre-processing and large-scale generative language model post-processing, which solves the problem of poor generalization of single models. By combining multiple methods, the accuracy and adaptability of tampering recognition are improved. In practical applications, this multi-path collaborative approach can effectively handle complex and diverse image tampering methods. The expert model pre-processing accurately captures potential tampering traces in the image, while the large-scale generative language model further performs in-depth analysis and logical reasoning on the detection results, compensating for potential misjudgments or omissions that may occur with single models in specific scenarios. Furthermore, this scheme possesses strong scalability, allowing for the flexible introduction of more advanced detection technologies and algorithms to adapt to constantly evolving image tampering methods. This multi-layered, multi-dimensional comprehensive decision-making mechanism enables the entire system to maintain high robustness and reliability even when facing various unknown tampering techniques.

[0045] This disclosure addresses the technical limitations of single-model image tampering detection. Specifically, relying solely on large-scale generative language models or expert models for tampering detection leads to decreased recognition rates when faced with small samples or data distributions varying significantly across different domains, resulting in inconsistent performance. Furthermore, traditional deep learning models heavily depend on high-quality, large-scale labeled data, and their "black box" nature makes the results lack interpretability, hindering the complete trust of reviewers. This disclosure proposes a fusion and complementary approach, allowing different models to collaborate in their strongest areas. This not only solves the problems of decreased recognition rates and limited application scenarios but also enhances interpretability, enabling reviewers to more clearly understand the model's decision-making process and thus increasing trust in the recognition results. By organically combining the advantages of expert models and large-scale generative language models, higher adaptability can be demonstrated in complex scenarios. For example, when dealing with subtle traces of image tampering, expert models can provide precise local feature analysis, while generative language models can further verify and supplement the expert model's results through logical deduction based on an understanding of global information. This collaborative mechanism not only improves identification accuracy but also significantly reduces false alarm and false negative rates. Furthermore, the technical solution is designed to fully consider the diversity and dynamic changes required in practical applications, ensuring rapid iteration and upgrades as new tampering methods emerge, and maintaining a consistently high level of detection capability.

[0046] In one exemplary embodiment, the method includes: performing attribute data consistency detection on the target image to obtain a consistency detection report; correspondingly, the step of performing a fusion decision on the trace detection report and the comparison report through a large generative language model to obtain a tampering identification result corresponding to the target image includes: performing a fusion decision on the trace detection report, the comparison report, and the consistency detection report through the large generative language model to obtain the tampering identification result.

[0047] This disclosure does not limit the attribute data detected during the attribute data consistency detection process. For example, attribute data may include image creation time, modification time, and timestamp information, and may also include image watermarks and digital signatures. Attribute data consistency detection refers to detecting whether this attribute data has been tampered with or is abnormal. For example, by comparing the creation time and modification time of an image file, it can be preliminarily determined whether the file has undergone post-processing. At the same time, the integrity verification of digital signature and watermark information can further confirm the authenticity and reliability of the image source. By inputting the result of attribute data consistency detection (consistency detection report), the trace detection report, and the comparison report into a large-scale generative language model for fusion decision-making, tampering identification results can be obtained. This provides a technical route that adopts multimodal fusion and triple collaboration, constructing a highly robust and highly interpretable image tampering identification scheme through three parallel and complementary analysis links.

[0048] In one exemplary implementation, please refer to Figure 2 The flowchart illustrates the tampering trace detection scheme of this disclosure. The step of performing tampering trace detection on the target image based on an expert model to obtain a trace detection report includes: S210. Perform at least one preprocessing operation on the target image to obtain the preprocessing result corresponding to each preprocessing operation.

[0049] This disclosure does not limit the preprocessing operations, but aims to improve the information quality of the expert model and / or the first inference model input below by actively enhancing the relevant features indicating tampering in the target image or by directly performing specific analysis on the target image to obtain sufficient information for extracting possible, imperceptible tampering traces in the target image. For example, the preprocessing operation on the target image includes performing at least one of the following: compression error accumulation analysis, wavelet transform, spectral analysis, multi-scale analysis, histogram equalization, and adaptive noise filtering. These preprocessing operations are described below: Accumulated Compression Error Analysis (ELA): By analyzing the cumulative errors generated during multiple compression processes, potential tampering can be detected. This technique is particularly suitable for detecting image files that have been saved or converted multiple times, as each compression introduces tiny errors, and these errors may manifest significantly differently in genuine and tampered images. For images containing only text (such as contracts), added or modified text can be identified by analyzing the differences in compression characteristics across different regions. The preprocessing result of accumulated compression error analysis can be a heatmap containing the accumulated error value for each pixel. This heatmap visually demonstrates the differences in compression characteristics across different regions of the image, providing crucial feature information for subsequent expert models.

[0050] Wavelet transform: By decomposing an image into sub-bands of different frequencies, wavelet transform can highlight local anomalous features in the image. This method is highly sensitive to detecting image tampering, especially for boundary discontinuities or texture inconsistencies introduced by operations such as splicing or copying and pasting. After wavelet transform processing, a coefficient distribution map reflecting the frequency characteristics of each region of the image can be generated. This distribution map can reveal potential tampering areas, such as abnormal edge information in the high-frequency part or unnatural smooth areas in the low-frequency part. These features can serve as important basis for subsequent analysis, further improving the ability of expert models to identify tampering traces. For example, this disclosure can use wavelet transform to perform multi-resolution decomposition of an image, which can effectively extract local features of the image. This method can help identify inconsistencies that may exist in the image, such as blurred edges. The significant advantage of wavelet transform is that it captures local anomalous features of the image at different scales and frequencies, and is particularly sensitive to minor splicing, copying, and other operations.

[0051] Spectral analysis: By performing spectral analysis on images, hidden information in the frequency domain can be further uncovered. This method transforms images from the spatial domain to the frequency domain, revealing anomalies that are difficult to detect in the spatial domain. For example, tampered areas may appear as irregular energy distributions or anomalous frequency component clusters in the spectral image. It can be used to detect periodic noise introduced by screen re-photographing, device differences, etc., or frequency domain artifacts that may be left by splicing tampering.

[0052] Multi-scale analysis: By decomposing and analyzing images at different scales, multi-scale analysis can capture both global and local features of an image. This method can effectively identify scale inconsistencies caused by tampering, such as anomalies that may be ignored at a coarse scale but become apparent at a fine scale. Combined with techniques such as image pyramids, this disclosure can analyze images at multiple resolution levels to obtain feature representations containing rich information. This multi-scale processing not only enhances sensitivity to subtle tampering traces but also helps distinguish between natural image changes and human modifications. By extracting and comparing features layer by layer, potential tampered areas can be located more accurately, providing a reliable basis for further verification.

[0053] Histogram equalization (HUE): By adjusting the grayscale distribution of an image, HUE can enhance image contrast, making hidden details more visible. This method is particularly effective when processing images with uneven lighting or low contrast, helping to reveal potential tampering traces. For example, after HUE processing, previously imperceptible edge transitions or local anomalies may become significant. Furthermore, this technique can be combined with other analysis methods to further enhance the detection capability of complex tampering behaviors. By optimizing the visual effect and feature representation of the image, HUE provides a more reliable data foundation for subsequent analysis steps. This disclosure utilizes HUE to enhance details in low-contrast areas, making tampering traces more obvious in shadowed or overexposed areas.

[0054] Adaptive noise filtering: Adaptive noise filtering is a technique that dynamically adjusts the filtering intensity based on the local characteristics of an image. This method effectively preserves image details while removing noise, preventing tampering traces from being masked by excessive smoothing. By analyzing the noise distribution characteristics of different regions of the image, adaptive noise filtering can specifically address abnormal areas, thereby highlighting potential tampering artifacts. For example, in stitching boundaries or cloned areas, noise patterns often differ from the surrounding environment; this technique can more clearly reveal these subtle differences. This disclosure utilizes adaptive noise filtering to intelligently separate the true content of an image from abnormal noise patterns that may be introduced by tampering.

[0055] S220. Using the expert model corresponding to the target image, perform fake region identification on each of the preprocessing results to obtain an image marked with the identification results.

[0056] This disclosure does not limit the preprocessing results, and different preprocessing operations may produce different results. For example, preprocessing results may include various forms of data representation, such as error accumulation heatmaps, frequency coefficient distribution maps, spectral energy distribution maps, multi-scale feature representations, and enhanced grayscale histograms. These results can provide rich information support for expert models, helping them to more accurately locate potential forgery areas. In practical applications, expert models combine multiple preprocessing results for comprehensive analysis to improve the robustness and accuracy of identification. For example, by combining compressed error accumulation analysis with wavelet transform results, abnormal features in the image can be captured simultaneously from both global and local perspectives. Furthermore, the results of spectral analysis and multi-scale analysis can complement each other, further revealing tampering traces hidden at different resolutions or frequencies. Finally, the expert model generates images labeled with the identification results based on these inputs, which may include highlighting of suspected forgery areas or other forms of annotation, thus providing a reliable basis for subsequent manual review or automated decision-making.

[0057] The image with the identified results refers to an image generated after analysis and processing by an expert model, which clearly marks potentially tampered areas or abnormal features. These images are typically visualized by combining the preprocessed results with the original image, creating a clear and intuitive display. For example, highlighted areas may indicate potentially forged parts, while other forms of marking are used to highlight specific types of anomalies. These markings not only help human reviewers quickly locate problematic areas but also provide clear reference points for automated systems, thereby improving overall detection efficiency and accuracy.

[0058] S230. Based on the natural language description of the difference between the target image and the image labeled with the recognition result, text description information is obtained.

[0059] This disclosure does not limit the method of obtaining textual description information based on natural language descriptions of the differences between the target image and the image labeled with recognition results. For example, the two images can be input into a multimodal large-scale generative language model, which automatically generates a detailed description of the differences based on the content and features of the two images. This description may include information such as the location, size, and shape of the tampered area, as well as possible tampering methods. Furthermore, the generated textual description information can be further combined with contextual semantic analysis to provide a deeper explanation, such as the possible causes or potential impacts of the tampering. In this way, not only can the understanding of the image content be enhanced, but it can also provide richer reference information for subsequent processing steps. In practical applications, this method has strong flexibility and scalability, and the generation strategy can be adjusted according to specific needs to meet the requirements of different scenarios. In other embodiments, textual description information can be generated by combining a preset template. For example, the preset template may contain a fixed structure and keywords, and the image difference information can be filled into the specified positions of the template to quickly generate standardized textual description information.

[0060] S240. Using a first inference model, the forgery traces of the target image are inferred based on the text description information to obtain the trace detection report. The first inference model is a large-scale generative language model.

[0061] For example, the text description information and the first prompt information can be input into the first inference model. The first prompt information prompts the first inference model to perform forgery trace inference on the target image based on the text description information and output the trace detection report. The first prompt information can provide structured information for the trace detection report, such as key steps in the inference process, key image areas of focus, basic report format requirements, and possible tampering categories. Furthermore, the first prompt information can also include specific constraints, such as priority settings for certain sensitive areas or attention weights for specific types of forgery methods, to further improve the accuracy and relevance of the inference results. To ensure the high usability of the output trace detection report, the first inference model comprehensively considers the details in the text description information and the guidance requirements in the first prompt information during the generation process, thereby achieving comprehensive and in-depth forgery trace inference.

[0062] In an exemplary embodiment, the first inference model is a multimodal large-scale generative language model. The step of inferring forgery traces from the target image based on the text description information using the first inference model to obtain the trace detection report includes: inferring forgery traces from the target image based on the source image and the text description information using the first inference model to obtain the trace detection report. The source image includes at least one of the following: the target image, an associated image obtained by preprocessing the target image, and the image marked with the recognition result.

[0063] This disclosure does not limit the associated image, and may include at least one of the following: an error accumulation heatmap, a frequency coefficient distribution map, a spectral energy distribution map, a multi-scale feature representation, and an enhanced grayscale histogram, etc. A trace detection report can be obtained by inputting the source image, text description information, and corresponding second prompt information into the first inference model. This disclosure does not limit the second prompt information; its function and design method are similar to the first prompt information, and will not be elaborated upon.

[0064] Please refer to Figure 3 The diagram illustrates a forgery trace reasoning process in an exemplary embodiment of this disclosure. Figure 3The image in the image is the target image. This target image undergoes six preprocessing operations: compression error accumulation analysis, wavelet transform, spectral analysis, multi-scale analysis, histogram equalization, and adaptive noise filtering. These preprocessing results are then input into an expert model for forgery region identification, resulting in an image labeled with the identification results. This labeled image and the target image can be used to determine text description information. This text description information, along with related source images, can be input into a first inference model (Large Language Model, LLM) for forgery trace inference. In some implementations, this first inference model can also be a Multimodal Large Language Model (MLLM). By inputting source images, text description information, and corresponding second prompt information into the first inference model (MLLM), the trace detection report can more accurately reflect the potential forgery features of the target image. This method fully utilizes the powerful inference capabilities of the multimodal large generative language model, combining multiple input information for comprehensive analysis, thereby improving the reliability and accuracy of the detection results. The first inference model can output... Figure 3 The analysis report in this document is the trace detection report. This disclosure does not limit the expert model; it can be a deep learning model that performs well in a specific vertical domain related to the target image. The training process of this expert model is not limited and does not constitute an obstacle to implementation. By converting the output of the expert model and other relevant information into natural language descriptions, and calling LLM for cross-dimensional logical reasoning, a highly readable analysis report is generated, and the output results of the expert model can be interpreted and verified.

[0065] In one exemplary embodiment, the trace detection report includes at least one of the following: Confidence level of forgery traces; Explanation of the reasoning process behind the forgery traces; If the forgery is confirmed, the proposed impact on the target business associated with the target image; If the confidence level of the forgery traces does not meet the preset confidence level requirement, other verification methods are proposed to determine whether the target image has been tampered with.

[0066] By outputting a trace detection report that includes this information, users can gain a more comprehensive understanding of the potential risks of the target image and the direction of subsequent processing. This report not only provides decision-makers with a scientific basis but also guides further verification work, thereby improving the efficiency and reliability of the overall detection process. At the same time, this structured output also provides flexibility for application scenarios in different fields, making the trace detection report more practical and operable.

[0067] For example, the target image is a screenshot of a text message, which includes the following text: Tuesday, December 17, 18:25 [Bank of China] Your credit card ending in **** was used for a purchase of 760.00 yuan on December 17th at 18:25. Your current available balance in RMB is 6692.53 yuan. For more repayment services, please click "Link 1" to view. For further details, please contact customer service.

[0068] Thursday, December 19th, 18:50 [Bank of China] Your credit card ending in **** received a deposit of 4293.08 yuan on December 19th at 21:16. After the deposit, your available balance is 10985.61 yuan. For details, please log in to "Link 2".

[0069] Using this screenshot as the target image, an expert model is invoked to obtain an image labeled with the recognition results. This image contains labels for four elements: "spending amount, deposit amount, available balance, and time." The trace detection report output by the first inference model includes the following: I. Summary of Findings The evidence shows that key numbers in the two bank text message screenshots are highlighted in several places, focusing on four elements: "spending amount, deposit amount, available balance, and time," suggesting that they have been modified or overwritten.

[0070] II. Detailed Analysis Description of abnormal areas Anomaly 1: In the first SMS bubble (below the top "Tuesday, December 17, 18:25"), the area of ​​the number "760.00" at "18:25 consumption of 760.00 yuan" is highlighted.

[0071] Anomaly 2: In the same bubble, the area of ​​the number "6692.53" in "Available balance is 6692.53 yuan" is continuously highlighted.

[0072] Anomaly 3: In the second SMS bubble (below "Thursday, December 19, 18:50"), the numbers "21:16" and "4293.08" in "21:16 deposit 4293.08 yuan" are highlighted and covered.

[0073] Anomaly 4: In the same bubble, there is a continuous area of ​​highlighted numbers in the "10985.61" area of ​​"Available Balance 10985.61 yuan".

[0074] Further observation: There was no equally strong highlighting near the short links "Link 1" and "Link 2" and the customer service phone number "&&&&&&&&" in the two text messages; the highlighting was mainly concentrated on the amount and time numbers, which is consistent with the distribution characteristics of "key fields being modified".

[0075] Inference of tampering methods Based on the fact that the highlighted distribution focuses on independent blocks of numbers and contrasts sharply with the surrounding unaltered text boundaries, one or a combination of the following techniques may be used: After partially smearing / covering and then overlaying new text: First, use a color similar to the background of the bubble to cover a small area, and then overlay the text with a similar font. This causes the high-frequency texture in that area to be discontinuous with the surrounding compressed noise, which is judged as abnormal by the model.

[0076] Copy-paste replacement: When numbers are captured from other screenshots / text messages and pasted in, inconsistencies in the frequency domain and edge artifacts may occur due to differences in compression rate, sharpness, and anti-aliasing methods of the source.

[0077] AI / intelligent inpainting tools can be used to repaint small areas and then fill in text: unnatural transitions appear at the edges of numbers, which is a common type of image repair mark.

[0078] Supporting logic: The highlights are close to the edges of the number strokes, distributed continuously in bands or blocks, and do not appear evenly throughout the entire text, ruling out false positives caused by "overall compression or shadows in the screenshot"; at the same time, the amount and balance are the most motivated targets for modification, which is consistent with the business scenario.

[0079] III. Overall Conclusion Confidence level of tampering: Medium-high Note: The anomalies are concentrated in four key numerical / time fields, and their spatial distribution is consistent with typical traces of numerical alteration. Although the numerical relationship between the two text messages appears to be self-consistent (6692.53 + 4293.08 = 10985.61), this "self-consistency" is more in line with artificially constructed consistency than evidence that excludes tampering.

[0080] Business Impact: If the above fields are altered, it will directly affect the assessment of the applicant's financial strength, transaction authenticity, and solvency. Based on the current evidence, it is not recommended to use this screenshot as valid bank statement / receipt proof. It is suggested to postpone or reject any evidence in this application that relies on this screenshot and proceed with in-depth manual verification.

[0081] IV. Recommendations Necessary verification of original vouchers You are required to submit an official electronic bank statement (PDF or CSV exported from online banking / mobile banking, including electronic signature or verification code), or a bank statement printed from a branch and stamped with a seal across the binding.

[0082] Screen recordings of the cardholder's original device operation are required: access the two text messages from the text message list, show the complete context and timeline, avoiding post-editing of single screenshots.

[0083] On-site (or video) verification: Log in to online banking / mobile banking to view transaction records and balance changes for the corresponding time period in real time.

[0084] Supplementary comparison for verifying the authenticity of text messages. Compare the details of the SMS messages sent by the operator (not the content, only the time and the sender) to verify whether there was indeed a bank notification at the corresponding time.

[0085] Verify that the customer service phone number and short link domain in the verification text message are from official channels; if they are inconsistent with official information, they should be considered high-risk materials.

[0086] Review Strategy In principle, the amount, time, and balance information in the screenshot should not be accepted until the original bank data that can be independently verified is obtained.

[0087] If the application relies on other pages / screenshots, it is recommended to carefully check all amount fields within the same time period and pay attention to whether there are any traces of the same type of partial rewriting.

[0088] This disclosed tampering trace detection scheme significantly improves the accuracy and efficiency of tampering trace detection by combining an expert model and a first inference model. The expert model can accurately identify forged regions based on the preprocessed results, ensuring high coverage and low false alarm rate for potentially tampered parts in the target image. Simultaneously, by utilizing a large-scale generative language model as the first inference model, textual descriptions of the forged region identification results can be obtained. In-depth analysis of these textual descriptions uncovers more subtle forgery clues, thereby generating a detailed and reliable trace detection report. This dual-model collaboration not only enhances the accuracy of the detection results but also enhances credibility due to the complete inference process. Since the first inference model is a large-scale generative language model, the synergy between the first inference model and the expert model fully leverages the advantages of multimodal data processing. Utilizing the powerful natural language understanding and generation capabilities of the large-scale generative language model, image features are organically combined with textual descriptions, enabling in-depth analysis of complex tampering behaviors. Furthermore, this scheme effectively reduces the error accumulation problem that may be caused by a single model through the division of labor and cooperation between the expert model and the first inference model, further improving the overall detection accuracy and robustness. In practical applications, this dual-model architecture also demonstrates strong adaptability, flexibly adjusting the detection strategy according to the needs of different scenarios, providing a more comprehensive and reliable solution for image tampering identification.

[0089] Please refer to Figure 4The diagram illustrates a reference image determination method proposed in this disclosure. Before retrieving at least one reference image similar to the target image and obtaining a comparison report by comparing the target image with the at least one reference image, the method includes: S410. Use a multimodal large-scale generative language model to extract features from the target image to obtain the corresponding features to be compared.

[0090] S420. Based on the feature to be compared, a search is performed in the historical feature database to obtain at least one reference feature similar to the feature to be compared, wherein the historical feature database is used to record features corresponding to historical images; and the image corresponding to each reference feature is used as the corresponding reference image.

[0091] The feature extraction method for historical images recorded in the historical feature database in this disclosure is the same as the feature extraction method in step S410. This disclosure does not limit the requirements, quantity, or acquisition method of the reference features. For example, a similarity threshold can be set according to actual needs to filter out reference features that meet the requirements. For instance, by calculating the Euclidean distance or cosine similarity between the feature to be compared and each feature in the historical feature database, several features with the smallest distance or the highest similarity can be selected as reference features. Furthermore, to improve retrieval efficiency, indexing techniques can be used to optimize the historical feature database, such as constructing an ND tree or using a locality-sensitive hashing algorithm, thereby achieving fast retrieval on large-scale datasets.

[0092] Please refer to Figure 5 The illustration shows a schematic diagram of the implementation process for determining a reference image in an exemplary embodiment of this disclosure.

[0093] The image in the diagram is the target image. This feature extraction process can utilize a multimodal large-scale generative language model pre-trained on massive datasets, encoding the image input to this model into a high-dimensional dense vector. This vector effectively captures the semantic information in the image and remains robust to common image transformations. Therefore, the features to be compared after feature extraction from the target image are a high-dimensional dense vector.

[0094] Specifically, the historical feature database is a vector database. Using this vector database, an efficient index structure is constructed from the feature vectors of all historical images. This design supports high-concurrency real-time queries, significantly reduces retrieval latency in cases with large-scale features, and completely avoids the high computational complexity of traditional pixel-by-pixel image retrieval methods.

[0095] Reference features can be determined through similarity comparison in vector databases. For example, the Top N candidate vectors closest to the feature to be compared can be quickly located in the vector database. For instance, L2 distance can be used as a similarity metric to accurately and efficiently find the most similar reference features and their corresponding reference images. This disclosure does not limit the value of N; in practical applications, the choice of N value usually depends on the specific needs of the scenario and the size of the dataset. A larger N value can provide more candidate features for further analysis, but it also increases computational overhead; while a smaller N value can reduce the interference of redundant information while ensuring efficiency. Therefore, the specific N value needs to be selected based on the actual situation. L2 distance refers to the Euclidean distance between two points in vector space, calculated as the square root of the sum of the squares of the differences between the corresponding dimensions of the two vectors. This metric can intuitively reflect the degree of difference between feature vectors, and is particularly suitable for similarity evaluation of high-dimensional dense vectors. Actual usage data demonstrates that vector indexing overcomes the performance bottleneck of traditional image comparison. When the feature library size is 80,000, the retrieval speed for a single image is 0.0024 seconds. This reference image retrieval method can accurately capture the core content of the image, effectively filter out irrelevant interference, reduce the false negative rate and false positive rate, and has the advantage of fast retrieval speed.

[0096] This disclosure proposes that traditional image query methods rely on pixel similarity comparison. With the growth of business volume, image databases rapidly reach tens of thousands, even millions or tens of millions of images. At this scale, traditional image query methods suffer from high computational complexity and low robustness. Specifically, traditional pixel-by-pixel comparison-based retrieval methods require matching the query image with all images in the database, resulting in a time complexity of O(N). When the image database reaches hundreds of thousands or even larger, the retrieval time increases dramatically, failing to meet real-time review requirements. Furthermore, images in actual business operations may undergo diverse processing such as rotation, cropping, compression, and brightness adjustment; relying solely on pixel-level matching yields low accuracy and is insufficient to meet review requirements.

[0097] Furthermore, related technologies have proposed image querying based on perceptual hashing algorithms. Perceptual hashing generates strings by performing discrete cosine transform on the image, and image queries are achieved through string comparison. The algorithm is simple and computationally fast. However, it is highly sensitive to pixel values, noise, and deformation of the image. Even slight cropping of an image can drastically change its hash value, leading to a high false negative rate and making it unsuitable for business scenarios requiring high robustness. In addition, some related technologies have proposed generating feature vectors based on traditional feature descriptors, such as ORB descriptors. These descriptors generate feature vectors with high dimensionality and complex matching processes. Their matching efficiency is low during large-scale retrieval, making it difficult to support real-time retrieval of ultra-large image databases. ORB descriptors refer to a keypoint-based feature description method that matches images by detecting keypoints and generating corresponding feature vectors. Although ORB descriptors can handle some image changes, such as rotation and scaling, their adaptability to complex situations such as lighting changes and compression distortion is weak, thus limiting their performance in real-time applications. Unlike these related technologies, this disclosure extracts high-dimensional dense features through a multimodal large-scale generative language model. These features are adaptable and robust to complex situations. Retrieval based on these features balances retrieval accuracy and speed, breaking through the bottleneck of traditional methods in related technologies.

[0098] In one exemplary embodiment, retrieving at least one reference image similar to the target image, and obtaining a comparison report by comparing the target image with the at least one reference image, includes: performing a difference comparison between the target image and each of the reference images based on a second inference model to obtain the comparison report, wherein the second inference model is a multimodal large generative language model.

[0099] This disclosure utilizes a multimodal large-scale generative language model to compare the target image with at least one reference image, generating a comparison report. This fully leverages the powerful semantic understanding and multimodal processing capabilities of the multimodal large-scale generative language model. This method not only compares images at the visual feature level but also incorporates contextual information and potential logical relationships to further improve accuracy. For example, in the scenario of verifying the authenticity of text messages, by comprehensively analyzing the text content, layout style, and background elements in the target and reference images, subtle signs of tampering can be effectively identified. Furthermore, this method supports the mining of implicit information within images, such as analyzing image metadata or pixel-level anomalies to aid in judgment. This approach demonstrates higher sensitivity and reliability when dealing with complex forgery methods, thus providing a more comprehensive and accurate solution for image tampering identification.

[0100] In one exemplary embodiment, the step of performing attribute data consistency detection on the target image to obtain a consistency detection report includes: extracting editing information of the software corresponding to the target image, wherein the editing information includes at least one of creation information and modification information; extracting timestamp information corresponding to the target image; verifying the consistency between the editing information and the timestamp information to obtain the consistency detection report. If the editing information and the timestamp information are inconsistent, it indicates that the target image may have been tampered with. This inconsistency is usually manifested as the creation time being later than the modification time, or a logical contradiction between the editing record and the timestamp. Through in-depth analysis of these attribute data, it is possible to further determine whether the image has been subject to human intervention or forgery. In addition, other detection methods can be combined, such as cross-validating device information and geographical location information in the image attribute data, for example, checking whether the device model recorded in the image attributes matches the actual shooting device, or analyzing whether there is an abnormal offset in the geographical location information. These additional verification steps can effectively enhance the credibility of the detection results. This method can provide more clues for tamper identification from the perspective of static analysis of attribute data.

[0101] Please refer to Figure 6 This illustration shows a schematic diagram of the image tampering recognition process in an exemplary embodiment of this disclosure. The image tampering recognition process includes three main components: Main Thread 1: Historical Image Comparison. Main Thread 1 describes the specific implementation steps of step S130. The historical image library refers to the image library corresponding to the historical feature library, used to extract reference images corresponding to the reference features. In Main Thread 1, LLM refers to the second inference model. TOPN objects refer to N reference images. This process aims to determine whether the target image involves forgery by comparing it with the N reference images, thereby generating a comparison report. By performing multi-dimensional comparison between the target image and archived images in the historical image library, abnormal areas such as additions, deletions, and replacements can be located, effectively identifying tampering traces caused by duplicate declarations and material reuse.

[0102] Main Thread Two: Metadata Information Analysis. Main Thread Two describes the specific implementation steps of "performing attribute data consistency detection on the target image and obtaining a consistency detection report." The metadata in this step refers to the editing information of the software corresponding to the target image. This process automatically extracts and parses the image's attribute data, detects key credibility indicators such as timestamp logical consistency, device source reliability, and modification history, identifies potential risks from the data source, and effectively combats low-level forgery and batch generation attacks.

[0103] Main Thread Three: Single Image Blind Detection. Main Thread Three describes the specific implementation steps of step S120. The AI-generated trace detection process involves the use of expert models and first-order inference models. This process can detect inherent tampering traces in an image based solely on a single image without relying on any prior information. These traces include, but are not limited to: inconsistent noise distribution, irregular texture breaks, recompression artifacts, abnormal color response, and AIGC-generated features, independently achieving accurate identification of highly forged images. AIGC-generated features refer to the unique patterns or traces left when content is generated through artificial intelligence. These features typically include specific pixel distribution patterns in the image, inherent biases of the generation model, and the algorithm's performance in handling details.

[0104] Finally, after decision fusion using LLM, the tampering identification result can be output, including the final judgment and a detailed diagnostic report. The image tampering identification method proposed in this disclosure has been widely applied in real-world scenarios, specifically identifying tampering in 91,349 images, identifying 1,860 suspicious images with an accuracy rate of over 95%, fully verifying the efficiency and reliability of the method.

[0105] Figure 7 This is a block diagram illustrating an image tampering detection device according to an exemplary embodiment. The device includes: The image acquisition module 710 is configured to acquire the target image to be identified as tampered with; The trace detection module 720 is configured to perform tamper trace detection on the target image based on an expert model and obtain a trace detection report; The comparison and retrieval module 730 is configured to perform a retrieval of at least one reference image similar to the target image, and to obtain a comparison report by comparing the target image with the at least one reference image; The fusion decision module 740 is configured to perform a fusion decision on the trace detection report and the comparison report through a large generative language model to obtain the tampering identification result corresponding to the target image.

[0106] In one exemplary embodiment, the apparatus further includes: The consistency detection module 750 is configured to perform attribute data consistency detection on the target image and obtain a consistency detection report. The fusion decision module 740 is configured to execute: The large-scale generative language model is used to fuse the trace detection report, the comparison report, and the consistency detection report to obtain the tampering identification result.

[0107] In one exemplary implementation, the trace detection module 720 is configured to perform: Perform at least one preprocessing operation on the target image to obtain the preprocessing result corresponding to each preprocessing operation; Using the expert model corresponding to the target image, the fake region is identified in each of the preprocessing results to obtain an image marked with the identification result; Textual description information is obtained based on the differences between the target image and the image labeled with the recognition results, using natural language description. The first inference model is used to infer forgery traces in the target image based on the text description information to obtain the trace detection report. The first inference model belongs to a large-scale generative language model.

[0108] In one exemplary implementation, the trace detection report includes at least one of the following: Confidence level of forgery traces; Explanation of the reasoning process behind the forgery traces; If the forgery is confirmed, the proposed impact on the target business associated with the target image; If the confidence level of the forgery traces does not meet the preset confidence level requirement, other verification methods are proposed to determine whether the target image has been tampered with.

[0109] In one exemplary implementation, the comparison and retrieval module 730 is configured to perform: A multimodal large-scale generative language model is used to extract features from the target image to obtain the corresponding features to be compared. Based on the feature to be compared, a search is performed in the historical feature database to obtain at least one reference feature similar to the feature to be compared. The historical feature database is used to record features corresponding to historical images. The image corresponding to each of the aforementioned reference features is used as the corresponding reference image.

[0110] In one exemplary implementation, the comparison and retrieval module 730 is configured to perform: Based on the second inference model, the target image is compared with each of the reference images to obtain the comparison report. The second inference model belongs to a multimodal large-scale generative language model.

[0111] In one exemplary implementation, the consistency detection module 750 is configured to perform: Extract the editing information of the software corresponding to the target image, wherein the editing information includes at least one of creation information and modification information; Extract the timestamp information corresponding to the target image; Verify the consistency between the edited information and the timestamp information to obtain the consistency detection report.

[0112] In one exemplary implementation, the preprocessing operation on the target image includes performing at least one of the following on the target image: Compression error accumulation analysis, wavelet transform, spectrum analysis, multi-scale analysis, histogram equalization, and adaptive noise filtering.

[0113] In one exemplary implementation, the first inference model is a multimodal large-scale generative language model, and the trace detection module 720 is configured to execute: Using the first inference model, based on the source image and the text description information, forgery trace inference is performed on the target image to obtain the trace detection report. The source image includes at least one of the following: the target image, an associated image obtained by preprocessing the target image, and the image marked with the recognition result.

[0114] Regarding the apparatus in the above embodiments, the specific manner of each step has been described in detail in the embodiments of the foregoing method, and will not be elaborated here.

[0115] Please refer to Figure 8 It illustrates the structural block of a computer device provided in an exemplary embodiment of this disclosure. Figure 1 The computer device can be a terminal. This computer device is used to implement the image tampering identification method provided in the above embodiments. Specifically: Typically, computer device 800 includes a processor 801 and a memory 802.

[0116] Processor 801 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 801 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 801 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In an exemplary embodiment, processor 801 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In an exemplary embodiment, processor 801 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0117] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In an exemplary embodiment, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one instruction, at least one program, code set, or instruction set, configured to be executed by one or more processors to implement the image tampering identification method described above.

[0118] In one exemplary embodiment, the computer device 800 may optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, memory 802, and peripheral device interface 803 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 803 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 804, a touch display screen 805, a camera assembly 806, an audio circuit 807, a positioning assembly 808, and a power supply 809.

[0119] Those skilled in the art will understand that Figure 8The structure shown does not constitute a limitation on the computer device 800, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0120] Please refer to Figure 9 It illustrates the structural block of a computer device provided in another exemplary embodiment of this disclosure. Figure 2 The computer device can be a server used to execute the image tampering detection method described above. Specifically: Computer device 900 includes a Central Processing Unit (CPU) 901, a system memory 904 including Random Access Memory (RAM) 902 and Read Only Memory (ROM) 903, and a system bus 905 connecting the system memory 904 and the CPU 901. Computer device 900 also includes a basic input / output system (I / O system) 906 that facilitates information transfer between various devices within the computer, and a mass storage device 907 for storing the operating system 913, application programs 914, and other program modules 911.

[0121] The basic input / output system 906 includes a display 908 for displaying information and an input device 909 for user input, such as a mouse or keyboard. Both the display 908 and the input device 909 are connected to the central processing unit 901 via an input / output controller 190 connected to the system bus 905. The basic input / output system 906 may also include the input / output controller 190 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 190 also provides output to a display screen, printer, or other types of output devices.

[0122] Mass storage device 907 is connected to central processing unit 901 via a mass storage controller (not shown) connected to system bus 905. Mass storage device 907 and its associated computer-readable media provide non-volatile storage for computer device 900. That is, mass storage device 907 may include computer-readable media (not shown) such as hard disk or CD-ROM (CompactDisc Read-Only Memory) drive.

[0123] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 904 and mass storage device 907 described above can be collectively referred to as memory.

[0124] According to various embodiments of this disclosure, the computer device 900 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 900 can be connected to a network 912 via a network interface unit 911 connected to a system bus 905, or the network interface unit 911 can be used to connect to other types of networks or remote computer systems (not shown).

[0125] The aforementioned memory also includes a computer program stored in the memory and configured to be executed by one or more processors to implement the aforementioned image tampering identification method.

[0126] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is executed by a processor to implement the image tampering identification method.

[0127] Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0128] In an exemplary embodiment, a computer-readable storage medium including program code is also provided, such as a memory including program code, which can be executed by a processor to complete the image tampering identification method described above. Optionally, the computer-readable storage medium may be read-only memory (ROM), random access memory (RAM), compact-disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0129] In an exemplary embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the image tampering identification method described above.

[0130] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0131] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for identifying image tampering, characterized in that, The method includes: Acquire the target image to be identified as tampered with; The target image is subjected to tampering trace detection based on an expert model, and a trace detection report is obtained; Retrieve at least one reference image that is similar to the target image, and obtain a comparison report by comparing the target image with the at least one reference image; A large-scale generative language model is used to fuse the trace detection report and the comparison report to obtain the tampering identification result corresponding to the target image.

2. The method according to claim 1, characterized in that, The method includes: Perform attribute data consistency detection on the target image to obtain a consistency detection report; The step of fusing the trace detection report and the comparison report using a large-scale generative language model to obtain the tampering identification result corresponding to the target image includes: The large-scale generative language model is used to fuse the trace detection report, the comparison report, and the consistency detection report to obtain the tampering identification result.

3. The method according to claim 1, characterized in that, The step of performing tampering trace detection on the target image based on an expert model to obtain a trace detection report includes: Perform at least one preprocessing operation on the target image to obtain the preprocessing result corresponding to each preprocessing operation; Using the expert model corresponding to the target image, the fake region is identified in each of the preprocessing results to obtain an image marked with the identification result; Textual description information is obtained based on the differences between the target image and the image labeled with the recognition results, using natural language description. The first inference model is used to infer forgery traces in the target image based on the text description information to obtain the trace detection report. The first inference model belongs to a large-scale generative language model.

4. The method according to claim 3, characterized in that, The trace detection report includes at least one of the following: Confidence level of forgery traces; Explanation of the reasoning process behind the forgery traces; If the forgery is confirmed, the proposed impact on the target business associated with the target image; If the confidence level of the forgery traces does not meet the preset confidence level requirement, other verification methods are proposed to determine whether the target image has been tampered with.

5. The method according to claim 1, characterized in that, Before retrieving at least one reference image similar to the target image and obtaining a comparison report by comparing the target image with the at least one reference image, the method includes: A multimodal large-scale generative language model is used to extract features from the target image to obtain the corresponding features to be compared. Based on the feature to be compared, a search is performed in the historical feature database to obtain at least one reference feature similar to the feature to be compared. The historical feature database is used to record features corresponding to historical images. The image corresponding to each of the aforementioned reference features is used as the corresponding reference image.

6. The method according to claim 5, characterized in that, The step of retrieving at least one reference image similar to the target image, and obtaining a comparison report by comparing the target image with the at least one reference image, includes: Based on the second inference model, the target image is compared with each of the reference images to obtain the comparison report. The second inference model belongs to a multimodal large-scale generative language model.

7. The method according to claim 2, characterized in that, The step of performing attribute data consistency detection on the target image to obtain a consistency detection report includes: Extract the editing information of the software corresponding to the target image, wherein the editing information includes at least one of creation information and modification information; Extract the timestamp information corresponding to the target image; Verify the consistency between the edited information and the timestamp information to obtain the consistency detection report.

8. The method according to claim 3, characterized in that, The preprocessing operation on the target image includes performing at least one of the following on the target image: Compression error accumulation analysis, wavelet transform, spectrum analysis, multi-scale analysis, histogram equalization, and adaptive noise filtering.

9. The method according to claim 3, characterized in that, The first inference model is a multimodal large-scale generative language model. The step of using the first inference model to infer forgery traces from the target image based on the text description information, and obtaining the trace detection report, includes: Using the first inference model, based on the source image and the text description information, forgery trace inference is performed on the target image to obtain the trace detection report. The source image includes at least one of the following: the target image, an associated image obtained by preprocessing the target image, and the image marked with the recognition result.

10. An image tampering detection device, characterized in that, The device includes: The image acquisition module is configured to acquire the target image to be identified as being tampered with; The trace detection module is configured to perform tamper trace detection on the target image based on an expert model and obtain a trace detection report; The comparison and retrieval module is configured to perform a search for at least one reference image that is similar to the target image, and to obtain a comparison report by comparing the target image with the at least one reference image; The fusion decision module is configured to perform a fusion decision on the trace detection report and the comparison report using a large generative language model to obtain the tampering identification result corresponding to the target image.

11. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the image tampering identification method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device performs the image tampering identification method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes a computer program stored in a readable storage medium, wherein at least one processor of a computer device reads from and executes the computer program, causing the device to perform the image tampering identification method as described in any one of claims 1 to 9.

Images