Image tampering detection method, device, storage medium and electronic device

Abstract

The present specification discloses an image tampering detection method, device, storage medium and electronic device, and relates to the field of artificial intelligence technology. The method fuses image color features with discrete cosine transform features so that the embodiments of the present specification can fuse two-dimensional information to perform tampering detection and judgment on the image to be detected, and based on the text position features of the target text, the interference of background color on text color is filtered out in the fused information.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an image tampering detection method, device, storage medium and electronic device. Background Art

[0002] In recent years, with the continuous development of image processing technology, the methods for modifying images have become more advanced and diversified. Currently, many scenarios require the review of material images uploaded by users, such as business scenarios, insurance processing scenarios, etc. However, in each scenario, there is a certain degree of material image tampering. When detecting whether the images approved in these scenarios have been tampered with, since these images usually contain text information, the background information of the image itself will interfere with the judgment of whether the text information in the image has been tampered with, resulting in the inability to accurately identify whether the image has been tampered with. Summary of the invention

[0003] This specification provides an image tampering detection method, device, storage medium and electronic device, which can solve the technical problem that the background information of the image itself will interfere with the judgment of whether the text information in the image has been tampered with, resulting in the inability to accurately identify whether the image has been tampered with. The technical solution is as follows:

[0004] In a first aspect, this specification provides an image tampering detection method, the method comprising:

[0005] Acquire an image to be detected, extract text position features of target text in the image to be detected, and extract image color features of the image to be detected based on the text position features; wherein the image color features include text color features and background color features;

[0006] Performing discrete cosine transform processing on the image to be detected, and extracting discrete cosine transform features based on the result of the discrete cosine transform processing;

[0007] The tampering result of the image to be detected is determined based on the image color feature and the discrete cosine transform feature.

[0008] In a second aspect, this specification provides an image tampering detection device, the device comprising:

[0009] An image color feature extraction module, adapted to obtain an image to be detected, extract text position features of target text in the image to be detected, and extract image color features of the image to be detected based on the text position features; wherein the image color features include text color features and background color features;

[0010] A discrete cosine transform feature extraction module, adapted to perform discrete cosine transform processing on the image to be detected, and extract discrete cosine transform features based on the result of the discrete cosine transform processing;

[0011] The semantic segmentation module is adapted to determine the tampering result of the image to be detected based on the image color feature and the discrete cosine transform feature.

[0012] In a third aspect, the present specification provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0013] In a fourth aspect, the present description provides a computer program product comprising instructions, which, when executed on a computer or a processor, causes the computer or the processor to execute the steps of the above method.

[0014] In a fifth aspect, this specification provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0015] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least: the interference of the background of the image to be detected on the detection of the target text can be filtered out by the text position feature, and then the image color feature of the image to be detected is extracted by using the text position feature, and the image color feature includes the text color feature and the background color feature; and the discrete cosine transform feature is extracted by performing discrete cosine transform processing on the image to be detected. Based on the image color feature, it can be determined whether there is a pixel value split between the text part and the background part of the image to be detected, and based on the discrete cosine transform feature, it can be determined whether there is compression artifact in the image to be detected. This specification combines the features of the two dimensions of image color feature and discrete cosine transform feature to determine whether the image to be detected has been tampered with, and at the same time, the interference of the background of the image to be detected on the detection of the target text is filtered out, so as to effectively solve the technical problem that the background information of the image itself will interfere with the judgment of whether the text information in the image has been tampered with, resulting in the inability to accurately identify whether the image has been tampered with. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in this specification or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1An exemplary system architecture diagram of an image tampering detection method provided by an exemplary embodiment of this specification;

[0018] Figure 2 A flowchart of an image tampering detection method provided by an exemplary embodiment of this specification;

[0019] Figure 3 A flowchart of another image tampering detection method provided as an exemplary embodiment of this specification;

[0020] Figure 4 A schematic diagram of an interface between a first image area and a second area provided by an exemplary embodiment of this specification;

[0021] Figure 5 A flowchart of another image tampering detection method provided by an exemplary embodiment of this specification;

[0022] Figure 6 A flowchart of a method for determining a fusion matrix provided by an exemplary embodiment of this specification;

[0023] Figure 7 A schematic diagram of a process for displaying a tampered area provided by an exemplary embodiment of this specification;

[0024] Figure 8 A schematic diagram of the architecture of an image tampering detection method provided by an exemplary embodiment of this specification;

[0025] Fig. 9 A schematic diagram of the structure of an image tampering detection device provided as an exemplary embodiment of this specification;

[0026] Fig.10 The present invention is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the features and advantages of one or more embodiments of this specification more obvious and easy to understand, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of one or more embodiments of this specification.

[0028] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Instead, they are only examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the attached claims.

[0029] In different scenarios of consumer finance, users are usually required to upload various types of materials for review. Here, the materials are often uploaded in the form of images. In the review process, verifying the authenticity of the materials is a very critical step. In various scenarios, there is a situation where some uploaded materials are forged through image processing technology, such as the forgery of storefront photos, merchant qualifications and other uploaded materials in merchant scenarios, the forgery of medical materials, poverty certificates, invoices and other uploaded materials in insurance and consumer finance scenarios, and the forgery of personal documents, business licenses and other uploaded materials in personal and institutional certification scenarios. These images usually contain text information, and the review of text information is very critical in these scenarios. The background information of the image itself will interfere with the judgment of whether the text information in the image has been tampered with, resulting in the inability to accurately identify whether the image has been tampered with.

[0030] Based on the above needs, one or more embodiments of this specification provide an image tampering detection method, first obtaining an image to be detected and extracting the text position features of the target text in the image to be detected. The position of the target text in the image to be detected can be determined by the text position features to filter out the interference of the background of the image to be detected on the detection of the target text, and the image color features of the image to be detected are extracted by the text position features, and the image color features include text color features and background color features; at the same time, the method also performs discrete cosine transform processing on the image to be detected, extracts discrete cosine transform features based on the results of the discrete cosine transform processing, and then determines the tampering result of the image to be detected by combining the image color features and the discrete cosine transform features.

[0031] Based on the image color features, it can be determined whether there is a pixel value split between the text part and the background part of the image to be detected. Based on the discrete cosine transform features, it can be determined whether there is compression artifacts in the image to be detected. This manual combines the two-dimensional features of image color features and discrete cosine transform features to determine whether the image to be detected has been tampered with, and at the same time filters out the interference of the background of the image to be detected on the detection of the target text, thereby effectively solving the technical problem that the background information of the image itself will interfere with the judgment of whether the text information in the image has been tampered with, resulting in the inability to accurately identify whether the image has been tampered with.

[0032] Figure 1An exemplary system architecture diagram of an image tampering detection method provided as an exemplary embodiment of this specification.

[0033] like Figure 1 As shown, the system architecture may include an electronic device 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the electronic device 101 and the server 103. The network 102 may include various types of wired communication links or wireless communication links, for example, the wired communication link includes an optical fiber, a twisted pair, or a coaxial cable, and the wireless communication link includes a Bluetooth communication link, a Wireless-Fidelity (Wi-Fi) communication link, or a microwave communication link.

[0034] The electronic device 101 can interact with the server 103 through the network 102 to receive messages from the server 103 or send messages to the server 103. The electronic device 101 can be hardware or software. When the electronic device 101 is hardware, it can be various electronic devices, including but not limited to smart watches, smart phones, tablet computers, laptop portable computers, and desktop computers. When the electronic device 101 is software, it can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules (for example: used to provide distributed services), or it can be implemented as a single software or software module, which is not specifically limited here.

[0035] Optionally, in one or more embodiments of the present specification, the electronic device 101 can respectively obtain the image to be detected, extract the text position features of the target text in the image to be detected, and extract the image color features of the image to be detected based on the text position features; wherein the image color features include text color features and background color features; perform discrete cosine transform processing on the image to be detected, and extract discrete cosine transform features based on the result of the discrete cosine transform processing; determine the tampering result of the image to be detected based on the image color features and the discrete cosine transform features.

[0036] The server 103 may be a server that provides various services. It should be noted that the server 103 may be hardware or software. When the server 103 is hardware, it may be implemented as a distributed server cluster consisting of multiple servers, or it may be implemented as a single server. When the server 103 is software, it may be implemented as multiple software or software modules (for example, for providing distributed services), or it may be implemented as a single software or software module, which is not specifically limited herein.

[0037] Alternatively, the system architecture may not include the server 103. In other words, the server 103 may be an optional device in the embodiments of this specification. That is, the method provided in the embodiments of this specification may be applied to a system structure that only includes the electronic device 101, and the embodiments of this application are not limited to this.

[0038] It should be understood that Figure 1 The number of electronic devices, networks and servers in the figure is only illustrative, and any number of electronic devices, networks and servers may be used according to implementation requirements.

[0039] See also Figure 2 , Figure 2 The present invention is a flowchart of an image tampering detection method provided by an exemplary embodiment of the present invention.

[0040] The execution subject of an exemplary embodiment of this specification may be an electronic device that executes the image tampering detection method, a processor in an electronic device that executes the image tampering detection method, or an image tampering detection service in an electronic device that executes the image tampering detection method. For the convenience of description, the specific execution process of the image tampering detection method is introduced below by taking the execution subject being a processor in an electronic device as an example.

[0041] like Figure 2 As shown, the image tampering detection method includes:

[0042] S202: Acquire an image to be detected, extract text position features of target text in the image to be detected, and extract image color features of the image to be detected based on the text position features; wherein the image color features include text color features and background color features.

[0043] To determine whether the image to be detected has been tampered with, first extract the text position features of the target text in the image to be detected. The text position features include the text position information of the target text in the image to be detected. Generally, the target text in the image to be detected can be the position information of all the text in the image to be detected; it can also be text in the image to be detected whose color is similar to the background color, that is, the target text can be text that may be interfered by the background color during the detection process of the image to be detected. The image color features of the image to be detected are extracted by the text position features, so that the text color features and background color features are extracted under the premise of filtering out the interference of the background in the image to be detected on the detection of the target text.

[0044] Here, the format of the image to be detected may be a JPEG (Joint Photographic Experts Group) format, so that the image to be detected can be subsequently processed by discrete cosine transform. The image to be detected includes but is not limited to a financial voucher image.

[0045] Optionally, obtaining the image to be detected in step S202 includes: obtaining the image to be detected that is uploaded by the user and awaits approval from the server.

[0046] The user uploads the image to be detected to be approved to the server, and the electronic device obtains the image to be detected from the server to perform image tampering detection on the image to be detected uploaded by the user. Here, since the image to be detected uploaded by the user to be approved can be obtained from the server, the acquisition process of the image to be detected is simplified, and there is no special requirement for the image to be detected, so that the application scope of the embodiments provided in this specification is also wider.

[0047] In a feasible implementation, when a merchant applies to join the platform, they generally upload images of storefront photos and business licenses to the platform's server through the online platform. After receiving these images from the platform's server, the electronic device begins to detect tampering of the images to determine whether the images have been tampered with.

[0048] There is also a feasible implementation method. When the insured applies for a claim to the insurance platform, he or she generally uploads images of medical materials, invoices, etc. to the platform's server through the online platform. After the electronic device receives these images from the platform's server, it begins to perform tampering detection on the images to determine whether the images have been tampered with.

[0049] Optionally, step S202 extracts text position features of the target text in the image to be detected, including: performing optical character recognition on the image to be detected to determine the text position of the target text in the image to be detected; and determining text position features corresponding to the text position based on the text position of the target text in the image to be detected.

[0050] By using an OCR (Optical Character Recognition) device to perform optical character recognition on the image to be detected, the characters in the image to be detected are detected, and their shapes are determined by detecting dark and light patterns. Then, the position of the target text in the image to be detected is determined to obtain the text position. The text position may include the contour position information of each target text and may also include the position information of each target text area. Afterwards, the coordinate information corresponding to the text position of each target text is converted into a mask to extract the text position feature corresponding to the text position.

[0051] S204: Perform discrete cosine transform processing on the image to be detected, and extract discrete cosine transform features based on the result of the discrete cosine transform processing.

[0052] Discrete Cosine Transform (DCT) approximates an image by summing a set of cosine functions of different frequencies and amplitudes. It is actually the real part of Fourier transform. Discrete cosine transform has an important property, that is, for an image, most of its visual information is concentrated in a few transform coefficients. DCT transform first transforms the image function into an even function form, and then performs a two-dimensional discrete Fourier transform on it. Therefore, DCT transform can be regarded as a simplified Fourier transform.

[0053] Optionally, performing discrete cosine transform processing on the image to be detected, and extracting discrete cosine transform features based on the result of the discrete cosine transform processing, including:

[0054] Firstly, discrete cosine transform is performed on the image to be detected to obtain discrete cosine transform coefficients and quantization table.

[0055] By performing discrete cosine transform processing on the image color information data of the image to be detected, corresponding discrete cosine transform coefficients and quantization tables are obtained.

[0056] Secondly, the discrete cosine transform coefficients and the quantization table are input into a preset discrete cosine transform information extraction module to extract discrete cosine transform features.

[0057] Here, the preset discrete cosine transform information extraction module may be a pre-established module for extracting discrete cosine transform features from discrete cosine transform coefficients and quantization tables, and the discrete cosine transform features are used to determine whether compression artifacts due to multiple compressions appear in the image to be detected. It should be noted that, generally, after being tampered with, an image may be compressed multiple times, so compression artifacts can be used as a basis for determining whether the image has been tampered with.

[0058] S206: Determine the tampering result of the image to be detected based on the image color features and discrete cosine transform features.

[0059] In the image color features, the interference of background color on text color has been filtered out based on the text position features. Then, the image color features and discrete cosine transform features are combined and fused to determine the tampering results of the image to be detected. The image color features can provide color clues of the image to be detected, such as the separation of pixel values ​​at the connection between the tampered part and the original image; the discrete cosine transform features can provide frequency domain information. After multiple compressions of the image to be detected, compression artifacts will be left in the discrete cosine transform features.

[0060] The fusion of image color features and discrete cosine transform features allows the embodiments of the present specification to fuse two dimensions of information to perform tampering detection and judgment on the image to be detected, and the fused information filters out the interference of background color on text color. Therefore, the embodiments of the present specification can effectively determine the tampering result of the image to be detected, even if the image to be detected is rich in text information.

[0061] See also Figure 3 , Figure 3 The present invention is a flowchart of another image tampering detection method provided by an exemplary embodiment of the present specification.

[0062] like Figure 3 As shown, in step S202, extracting the image color features of the image to be detected based on the text position features includes:

[0063] S302: Determine a first image region of target text in the image to be detected and a remaining second image region in the image to be detected based on the text position feature.

[0064] The text position feature includes the position information of the target text, and the image to be detected can be divided into a first image area of ​​the target text based on the text position feature. For example, the first image area of ​​the target text can be divided based on the font edge position information of the target text. At the same time, the part of the image to be detected except the first image area can be used as the second image area.

[0065] like Figure 4 As shown, Figure 4 The interface diagram of the first image area and the second area provided by an exemplary embodiment of this specification. The dotted areas A1, A2, A3 and A4 corresponding to XXXX in the figure are all the first image areas, where XXXX represents the target text, and the remaining area B1 is the second image area.

[0066] S304: Extracting text color features and background color features of the image to be detected based on the first image region and the second image region respectively.

[0067] The text color features of the target text are extracted from the first image area of ​​the image to be detected, and the text color features are used to provide color clues for the target text; the background color features are extracted from the second image area of ​​the image to be detected, and the background color features are used to provide color clues for the background of the image to be detected. In this way, the background color features are effectively filtered out from the text color features to avoid interference with the text color features.

[0068] S306: Obtaining image color features of the image to be detected based on the text color features and the background color features.

[0069] The text color features and the background color features are combined to form the image color features of the image to be detected. Such image color features effectively distinguish the text color features from the background color features, so that when the image to be detected is rich in text information, it can effectively identify whether the text in the image to be detected has been tampered with, and then identify whether the image to be detected has been tampered with.

[0070] See also Figure 5 , Figure 5 The present invention is a flowchart of another method for detecting image tampering provided by an exemplary embodiment of the present invention.

[0071] like Figure 5 As shown, in step S206, determining the tampering result of the image to be detected based on the image color feature and the discrete cosine transform feature includes:

[0072] S502: Determine an image color matrix corresponding to the image color feature based on the image color feature, and determine a discrete cosine transform matrix corresponding to the discrete cosine transform feature based on the discrete cosine transform feature.

[0073] The preset image information extraction module is used to extract the image color features of the image to be detected and convert the image color features into an image color matrix. The preset discrete cosine transform information extraction module is used to extract discrete cosine transform features from the discrete cosine transform coefficients and quantization table corresponding to the image to be detected, and the discrete cosine transform features are converted into a discrete cosine transform matrix.

[0074] S504: Fusing the image color matrix with the discrete cosine transform matrix to obtain a fusion matrix corresponding to the image to be detected; wherein the value of each element in the fusion matrix represents the confidence value of the pixel point at the corresponding position in the image to be detected.

[0075] The image color matrix and the discrete cosine transform matrix are fused by using a preset information fusion module, and the length × width of the fused matrix is ​​processed so that the length × width of the fused matrix is ​​the same as the length × width of the pixel point of the image to be detected. Here, the value of each element in the fusion matrix represents the confidence value of the pixel point at the corresponding position in the image to be detected. Each element in the fusion matrix fuses the image color features and discrete cosine transform features of the pixel point at the corresponding position in the image to be detected, and the confidence value filters out the interference of the background in the image to be detected on the detection of the target text. Therefore, the confidence value corresponding to the pixel point obtained based on these two features is more accurate.

[0076] S506: Input the fusion matrix corresponding to the image to be detected into a preset image tampering classification module to determine the tampering result of the image to be detected.

[0077] By inputting the fusion matrix into the preset image tampering classification module, the preset image tampering classification module learns and fuses the confidence values ​​corresponding to each pixel in the image to be detected to determine the tampering result of the image to be detected, that is, to determine whether the image to be detected has been tampered with or not.

[0078] Optionally, inputting the fusion matrix corresponding to the image to be detected into a preset image tampering classification module to determine the tampering result of the image to be detected includes:

[0079] The fusion matrix corresponding to the image to be detected is input into the preset image tampering classification module to obtain the image confidence value corresponding to the image to be detected. If the image confidence value is greater than or equal to the preset image confidence value, the tampering result of the image to be detected is tampered.

[0080] After the fusion matrix corresponding to the image to be detected is input into the preset image tampering classification module, the preset image tampering classification module processes the fusion matrix. The preset image tampering classification module determines the confidence value corresponding to each pixel point in the image to be detected based on the image color features and discrete cosine transform features, and obtains the image confidence value corresponding to the entire image to be detected. The higher the image confidence value, the lower the probability of tampering of the image to be detected.

[0081] Here, the preset image confidence value may be preset in the preset image tampering classification module. The image preset confidence value may be determined through a large amount of training, such as inputting a large number of tampered images and a large number of untampered images into the preset image tampering classification module to obtain image confidence values ​​corresponding to a large number of tampered images and image confidence values ​​corresponding to a large number of untampered images, and then determining the dividing point between the image confidence values ​​corresponding to a large number of tampered images and the image confidence values ​​corresponding to a large number of untampered images, thereby determining the image preset confidence value.

[0082] Therefore, after obtaining the image confidence value corresponding to the entire image to be detected, the image confidence value is compared with the preset image confidence value. When the image confidence value is greater than or equal to the preset image confidence value, it can be judged that the tampering result of the image to be detected is tampered with; when the image confidence value is less than the preset image confidence value, it can be judged that the tampering result of the image to be detected is not tampered with.

[0083] See also Figure 6 , Figure 6 A flowchart of a method for determining a fusion matrix is ​​provided as an exemplary embodiment of this specification.

[0084] like Figure 6 As shown, in step S506, the image color matrix is ​​fused with the discrete cosine transform matrix to obtain a fusion matrix corresponding to the image to be detected, including:

[0085] S602: Input the image color matrix and the discrete cosine transform matrix into a preset information fusion module to obtain a target matrix.

[0086] Based on the preset information fusion module, the image color features and discrete cosine transform features are learned from the image color matrix and the discrete cosine transform matrix respectively, and then the image color matrix and the discrete cosine transform matrix are fused to obtain a target matrix with both image color features and discrete cosine transform features.

[0087] S604: Input the target matrix into the high-resolution information extraction module to extract fine-grained features, and determine a fine-grained matrix corresponding to the target matrix based on the fine-grained features.

[0088] After the target matrix is ​​input into the high-resolution information extraction module, the target matrix is ​​enhanced by the high-resolution information extraction module to obtain multiple enhanced data, which are then integrated with the target matrix to obtain fine-grained features. The fine-grained matrix integrates the image color features and discrete cosine transform features of smaller areas in the image to be detected. Therefore, based on the fine-grained matrix, more detailed original image information in the image to be detected can be obtained, and smaller tampering traces in the image to be detected can be captured.

[0089] S606: Perform a weighted average of the first element in the target matrix corresponding to each pixel in the image to be detected and the second element in the fine-grained matrix to obtain a weighted average value of the target corresponding to each pixel in the image to be detected, and determine the fusion matrix based on the weighted average value of the target corresponding to each pixel in the image to be detected.

[0090] The first element in the target matrix can represent the confidence value of the image color features and discrete cosine transform features, and the second element in the fine-grained matrix can represent the confidence value of the image color features and discrete cosine transform features of a smaller area in the image to be detected. The two confidence values ​​are weighted averaged so that each element in the fusion matrix can take into account the confidence values ​​of the image color features and discrete cosine transform features as well as the confidence values ​​of the image color features and discrete cosine transform features of a smaller area in the image to be detected, so that the fusion matrix can obtain more fine-grained features in the image to be detected. Here, the weighting coefficient of the first element and the weighting coefficient of the second element can be preset artificially. For example, the weighting coefficient of the first element can be 0.5, and the weighting coefficient of the second element can be 0.5; or the weighting coefficient of the first element can be 0.7, and the weighting coefficient of the second element can be 0.3, etc.

[0091] See also Figure 7 , Figure 7 A schematic diagram of a flow chart of displaying a tampered area is provided as an exemplary embodiment of this specification.

[0092] like Figure 7 As shown, after the fusion matrix corresponding to the image to be detected is input into the preset image tampering classification module to determine the tampering result of the image to be detected, it also includes:

[0093] S702: When the tampering result of the image to be detected is tampered, determine the target element in the fusion matrix whose value is greater than or equal to the preset confidence value of the pixel point, and determine the tampered target pixel point in the image to be detected based on the target element.

[0094] After the preset image tampering classification module determines that the image to be detected has been tampered with, each element in the fusion matrix is ​​compared with the preset confidence value of the pixel point, and the element in the fusion matrix that is smaller than the preset confidence value of the pixel point is used as the target element. The position of the target pixel point in the image to be detected is determined by the position of the target element. Since the pixel point confidence value corresponding to the target pixel point is smaller than the preset confidence value of the pixel point, the target pixel point is determined to be a tampered pixel point.

[0095] S704: Based on the target pixel point, differentially display the tampered area of ​​the image to be detected.

[0096] Through the position information of each target pixel point, each target pixel point is determined, and the area formed by each target pixel point is used as the tampered area, and then the tampered area formed by the target pixel point and the area formed by the remaining pixels are displayed differentially to display the tampered area of ​​the image to be detected. For example, the tampered area can be highlighted, blurred, and marked for identification to differentiate the tampered area of ​​the image to be detected.

[0097] In an embodiment provided in this specification, determining the first image region of the target text in the image to be detected and the remaining second image region in the image to be detected based on the text position feature in step S302 includes:

[0098] Inputting the text position feature and the image to be detected into a preset image information extraction module to determine a first image area of ​​the target text in the image to be detected and a remaining second image area in the image to be detected;

[0099] In step S304, respectively extracting text color features and background color features of the image to be detected based on the first image area and the second image area includes:

[0100] Based on the first image area and the second image area, the text color features and the background color features of the image to be detected are extracted by a preset image information extraction module.

[0101] The preset image information extraction module determines the first image area corresponding to the target text in the image to be detected and the remaining second image area in the image to be detected according to the text position feature and the image to be detected. Afterwards, the preset image information extraction module extracts the text color feature and the background color feature of the image to be detected according to the determined first image area and second image area.

[0102] Optionally, before inputting the text position feature and the image to be detected into a preset image information extraction module, the method further includes:

[0103] Data enhancement processing is performed on the image to be detected to obtain the image to be detected after data enhancement.

[0104] Data enhancement processing of the image to be detected includes rotating, dithering, and other processing of the image to be detected to increase the richness of the sample data of the image to be detected, so that the subsequent preset image information extraction module can better learn the characteristics of the image to be detected.

[0105] See also Figure 8 , Figure 8 The present invention is a schematic diagram of the architecture of an image tampering detection method provided by an exemplary embodiment of the present invention.

[0106] First, the data preprocessing part includes: inputting the image to be detected into the OCR network to extract the text position features of the target text, and at the same time, the image to be detected is subjected to data enhancement processing through the data enhancement module so that the preset image information extraction module can learn more features of the image to be detected, and the preset image information extraction module can filter out the interference of the background on the target text based on the text position features; the image preprocessing module obtains the corresponding discrete cosine transform features and quantization table by performing discrete cosine transform processing on the image to be detected, and then inputs the discrete cosine transform features and quantization table into the preset discrete cosine transform information extraction module; the preset image information extraction module and the preset discrete cosine transform information extraction module each output a matrix with feature information.

[0107] The semantic segmentation part includes: inputting these two matrices with feature information into the preset information fusion module, which will output a matrix with a length and width of one quarter of the image to be detected to reduce the amount of calculation, and then the matrix output by the preset information fusion module is deconvolved and passed to the high-resolution information extraction module, which outputs a matrix with a length and width of one-half of the image to be detected, and upsampling the matrix output by the preset information fusion module so that the length and width of the matrix are both one-half of the image to be detected, and finally weighted averaging the matrix output by the high-resolution information extraction module and the matrix output by the preset information fusion module with a processed length and width of one-half of the image to be detected. Based on the tampering classification result, it is determined whether to display the tampered area of ​​the image to be detected differently.

[0108] The tampering classification part includes: upsampling the matrix obtained by weighted averaging and inputting it into a preset image tampering classification model to obtain a tampering classification result.

[0109] Here, the data enhancement module, the preset image information extraction module, the image preprocessing module, the preset discrete cosine transform information extraction module, the preset information fusion module, the high-resolution information extraction module and the preset image tampering classification model can be their respective corresponding neural network modules.

[0110] See also Fig. 9 , Fig. 9 The present invention is a schematic structural diagram of an image tampering detection device provided by an exemplary embodiment of the present invention.

[0111] like Fig. 9 As shown, the image tampering detection device 900 includes:

[0112] The image color feature extraction module 910 is adapted to obtain an image to be detected, extract text position features of target text in the image to be detected, and extract image color features of the image to be detected based on the text position features; wherein the image color features include text color features and background color features;

[0113] A discrete cosine transform feature extraction module 920, adapted to perform discrete cosine transform processing on the image to be detected, and extract discrete cosine transform features based on the result of the discrete cosine transform processing;

[0114] The semantic segmentation module 930 is adapted to determine the tampering result of the image to be detected based on the image color features and discrete cosine transform features.

[0115] Optionally, the image color feature extraction module 910 is further adapted to determine a first image region of the target text in the image to be detected and a remaining second image region in the image to be detected based on the text position feature;

[0116] Extracting text color features and background color features of the image to be detected based on the first image area and the second image area respectively;

[0117] Based on the text color features and the background color features, the image color features of the image to be detected are obtained.

[0118] Optionally, the discrete cosine transform feature extraction module 920 is further adapted to perform discrete cosine transform processing on the image to be detected to obtain discrete cosine transform coefficients and a quantization table;

[0119] The discrete cosine transform coefficients and the quantization table are input into a preset discrete cosine transform information extraction module to extract discrete cosine transform features.

[0120] Optionally, the semantic segmentation module 930 is further adapted to determine an image color matrix corresponding to the image color feature based on the image color feature, and to determine a discrete cosine transform matrix corresponding to the discrete cosine transform feature based on the discrete cosine transform feature;

[0121] The image color matrix is ​​fused with the discrete cosine transform matrix to obtain a fusion matrix corresponding to the image to be detected; wherein the value of each element in the fusion matrix represents the confidence value of the pixel point at the corresponding position in the image to be detected;

[0122] The fusion matrix corresponding to the image to be detected is input into a preset image tampering classification module to determine the tampering result of the image to be detected.

[0123] Optionally, the semantic segmentation module 930 is further adapted to input the image color matrix and the discrete cosine transform matrix into a preset information fusion module to obtain a target matrix;

[0124] Inputting the target matrix into a high-resolution information extraction module to extract fine-grained features, and determining a fine-grained matrix corresponding to the target matrix based on the fine-grained features;

[0125] The first element in the target matrix corresponding to each pixel in the image to be detected and the second element in the fine-grained matrix are weighted averaged to obtain the weighted average value of the target corresponding to each pixel in the image to be detected, and the fusion matrix is ​​determined based on the weighted average value of the target corresponding to each pixel in the image to be detected.

[0126] Optionally, the semantic segmentation module 930 is also suitable for inputting the fusion matrix corresponding to the image to be detected into a preset image tampering classification module to obtain an image confidence value corresponding to the image to be detected. If the image confidence value is greater than or equal to the preset image confidence value, the tampering result of the image to be detected is tampered.

[0127] Optionally, the semantic segmentation module 930 is further adapted to, when the tampering result of the image to be detected is tampered, determine a target element in the fusion matrix whose value is greater than or equal to a preset confidence value of the pixel point, and determine the tampered target pixel point in the image to be detected based on the target element;

[0128] Based on the target pixels, the tampered areas of the image to be detected are displayed differentially.

[0129] Optionally, the image color feature extraction module 910 is also adapted to perform optical character recognition on the image to be detected to determine the text position of the target text in the image to be detected;

[0130] Based on the text position of the target text in the image to be detected, a text position feature corresponding to the text position is determined.

[0131] Optionally, the image color feature extraction module 910 is also adapted to obtain the image to be detected that is uploaded by the user and is awaiting approval from the server.

[0132] Optionally, the image color feature extraction module 910 is further adapted to input the text position feature and the image to be detected into a preset image information extraction module to determine a first image region of the target text in the image to be detected and a remaining second image region in the image to be detected;

[0133] Extracting text color features and background color features of the image to be detected based on the first image area and the second image area respectively includes:

[0134] Based on the first image area and the second image area, the text color features and the background color features of the image to be detected are extracted by a preset image information extraction module.

[0135] Optionally, the image color feature extraction module 910 is further adapted to perform data enhancement processing on the image to be detected to obtain the image to be detected after data enhancement.

[0136] One or more embodiments of the present specification further provide a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the steps of any of the methods in the above embodiments.

[0137] One or more embodiments of the present specification also provide a computer program product comprising instructions, and when the computer program product is executed on a computer or a processor, the computer or the processor executes the steps of any one of the methods in the above embodiments.

[0138] For further information, see Fig.10 , Fig.10 This is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of this specification. Fig.10 As shown, the electronic device 1000 may include: at least one processor 1001 , at least one network interface 1004 , a user interface 1003 , a memory 1005 , and at least one communication bus 1002 .

[0139] The communication bus 1002 is used to realize the connection and communication between these components.

[0140] The user interface 1003 may include a display screen (Display) and a camera (Camera), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.

[0141] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0142] Among them, the processor 1001 may include one or more processing cores. The processor 1001 uses various interfaces and lines to connect the various parts of the entire electronic device 1000, and executes various functions and processes data of the electronic device 1000 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Optionally, the processor 1001 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 1001 can integrate one or more combinations of processors (Central Processing Unit, CPU), image processors (Graphics Processing Unit, GPU) and modems. Among them, the CPU mainly processes operating systems, user interfaces, and applications; the GPU is responsible for rendering and drawing the content that needs to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 1001, and it can be implemented separately through a chip.

[0143] Among them, the memory 1005 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1005 may optionally be at least one storage device located away from the aforementioned processor 1001. As Fig.10 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an image tampering detection program.

[0144] exist Fig.10In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 1001 can be used to call the image tampering detection program stored in the memory 1005 and specifically perform the following operations:

[0145] Acquire an image to be detected, extract text position features of target text in the image to be detected, and extract image color features of the image to be detected based on the text position features; wherein the image color features include text color features and background color features;

[0146] Perform discrete cosine transform processing on the image to be detected, and extract discrete cosine transform features based on the result of discrete cosine transform processing;

[0147] The tampering result of the image to be detected is determined based on the image color features and discrete cosine transform features.

[0148] Optionally, extracting image color features of the image to be detected based on the text position features includes:

[0149] Determine a first image region of the target text in the image to be detected and a remaining second image region in the image to be detected based on the text position feature;

[0150] Extracting text color features and background color features of the image to be detected based on the first image area and the second image area respectively;

[0151] Based on the text color features and the background color features, the image color features of the image to be detected are obtained.

[0152] Optionally, performing discrete cosine transform processing on the image to be detected, and extracting discrete cosine transform features based on the result of the discrete cosine transform processing, including:

[0153] Perform discrete cosine transform processing on the image to be detected to obtain discrete cosine transform coefficients and a quantization table;

[0154] The discrete cosine transform coefficients and the quantization table are input into a preset discrete cosine transform information extraction module to extract discrete cosine transform features.

[0155] Optionally, determining the tampering result of the image to be detected based on the image color feature and the discrete cosine transform feature includes:

[0156] Determine an image color matrix corresponding to the image color feature based on the image color feature, and determine a discrete cosine transform matrix corresponding to the discrete cosine transform feature based on the discrete cosine transform feature;

[0157] The image color matrix is ​​fused with the discrete cosine transform matrix to obtain a fusion matrix corresponding to the image to be detected; wherein the value of each element in the fusion matrix represents the confidence value of the pixel point at the corresponding position in the image to be detected;

[0158] The fusion matrix corresponding to the image to be detected is input into a preset image tampering classification module to determine the tampering result of the image to be detected.

[0159] Optionally, the image color matrix is ​​fused with the discrete cosine transform matrix to obtain a fusion matrix corresponding to the image to be detected, including:

[0160] The image color matrix and the discrete cosine transform matrix are input into a preset information fusion module to obtain a target matrix;

[0161] Inputting the target matrix into a high-resolution information extraction module to extract fine-grained features, and determining a fine-grained matrix corresponding to the target matrix based on the fine-grained features;

[0162] The first element in the target matrix corresponding to each pixel in the image to be detected and the second element in the fine-grained matrix are weighted averaged to obtain the weighted average value of the target corresponding to each pixel in the image to be detected, and the fusion matrix is ​​determined based on the weighted average value of the target corresponding to each pixel in the image to be detected.

[0163] Optionally, inputting the fusion matrix corresponding to the image to be detected into a preset image tampering classification module to determine the tampering result of the image to be detected includes:

[0164] The fusion matrix corresponding to the image to be detected is input into the preset image tampering classification module to obtain the image confidence value corresponding to the image to be detected. If the image confidence value is greater than or equal to the preset image confidence value, the tampering result of the image to be detected is tampered.

[0165] Optionally, after the fusion matrix corresponding to the image to be detected is input into a preset image tampering classification module to determine the tampering result of the image to be detected, the method further includes:

[0166] When the tampering result of the image to be detected is tampered, determine the target element whose value is greater than or equal to the preset confidence value of the pixel in the fusion matrix, and determine the tampered target pixel in the image to be detected based on the target element;

[0167] Based on the target pixels, the tampered areas of the image to be detected are displayed differentially.

[0168] Optionally, extracting text position features of target text in the image to be detected includes:

[0169] Perform optical character recognition on the image to be detected to determine the text position of the target text in the image to be detected;

[0170] Based on the text position of the target text in the image to be detected, a text position feature corresponding to the text position is determined.

[0171] Optionally, obtaining the image to be detected includes: obtaining the image to be detected for approval uploaded by the user from a server.

[0172] Optionally, determining a first image region of target text in the image to be detected and a remaining second image region in the image to be detected based on the text position feature includes:

[0173] Inputting the text position feature and the image to be detected into a preset image information extraction module to determine a first image area of ​​the target text in the image to be detected and a remaining second image area in the image to be detected;

[0174] Extracting text color features and background color features of the image to be detected based on the first image area and the second image area respectively includes:

[0175] Based on the first image area and the second image area, the text color features and the background color features of the image to be detected are extracted by a preset image information extraction module.

[0176] Optionally, before inputting the text position feature and the image to be detected into a preset image information extraction module, the method further includes:

[0177] Data enhancement processing is performed on the image to be detected to obtain the image to be detected after data enhancement.

[0178] One or more embodiments of the present specification also provide an image tampering detection system, including any one or more of the above-mentioned servers and any one or more of the above-mentioned electronic devices.

[0179] In one or more embodiments of this specification, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0180] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The above computer program product includes one or more computer instructions. When the above computer program instructions are loaded and executed on a computer, the above process or function according to the embodiment of this specification is generated in whole or in part. The above computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above computer instructions can be stored in a computer-readable storage medium or transmitted by the above computer-readable storage medium. The above computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The above computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The above-mentioned available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, digital versatile discs (DVD)), or semiconductor media (for example, solid state disks (SSD)), etc.

[0182] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that one or more embodiments of this specification are not limited by the order of the actions described, because according to one or more embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for one or more embodiments of this specification.

[0183] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0184] The above is a description of an image tampering detection method, device, storage medium and electronic device provided by one or more embodiments of this specification. For technicians in this field, according to the ideas of one or more embodiments of this specification, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on one or more embodiments of this specification.

Claims

1. A method for detecting image tampering, the method comprising: Acquire an image to be detected, extract text position features of target text in the image to be detected, and extract image color features of the image to be detected based on the text position features; wherein the image color features include text color features and background color features; Performing discrete cosine transform processing on the image to be detected, and extracting discrete cosine transform features based on the result of the discrete cosine transform processing; Determine an image color matrix corresponding to the image color feature based on the image color feature, and determine a discrete cosine transform matrix corresponding to the discrete cosine transform feature based on the discrete cosine transform feature; The image color matrix is ​​fused with the discrete cosine transform matrix to obtain a fusion matrix corresponding to the image to be detected; wherein the value of each element in the fusion matrix represents the confidence value of the pixel point at the corresponding position in the image to be detected; The fusion matrix corresponding to the image to be detected is input into a preset image tampering classification module to determine the tampering result of the image to be detected.

2. According to the image tampering detection method of claim 1, the step of extracting the image color feature of the image to be detected based on the text position feature comprises: Determine a first image region of the target text in the image to be detected and a remaining second image region in the image to be detected based on the text position feature; Extracting text color features and background color features of the image to be detected based on the first image area and the second image area respectively; Based on the text color features and the background color features, the image color features of the image to be detected are obtained.

3. According to the image tampering detection method of claim 1, the discrete cosine transform processing is performed on the image to be detected, and the discrete cosine transform feature is extracted based on the result of the discrete cosine transform processing, comprising: Performing discrete cosine transform processing on the image to be detected to obtain discrete cosine transform coefficients and a quantization table; The discrete cosine transform coefficients and the quantization table are input into a preset discrete cosine transform information extraction module to extract discrete cosine transform features.

4. According to the image tampering detection method of claim 1, the step of fusing the image color matrix with the discrete cosine transform matrix to obtain a fusion matrix corresponding to the image to be detected comprises: Input the image color matrix and the discrete cosine transform matrix into a preset information fusion module to obtain a target matrix; Inputting the target matrix into a high-resolution information extraction module to extract fine-grained features, and determining a fine-grained matrix corresponding to the target matrix based on the fine-grained features; A weighted average is performed on the first element in the target matrix corresponding to each pixel point in the image to be detected and the second element in the fine-grained matrix to obtain a weighted average value of the target corresponding to each pixel point in the image to be detected, and a fusion matrix is ​​determined based on the weighted average value of the target corresponding to each pixel point in the image to be detected.

5. The image tampering detection method according to claim 4, inputting the fusion matrix corresponding to the image to be detected into a preset image tampering classification module to determine the tampering result of the image to be detected, comprising: The fusion matrix corresponding to the image to be detected is input into a preset image tampering classification module to obtain an image confidence value corresponding to the image to be detected. If the image confidence value is greater than or equal to the preset image confidence value, the tampering result of the image to be detected is tampered.

6. The image tampering detection method according to claim 5, after inputting the fusion matrix corresponding to the image to be detected into a preset image tampering classification module to determine the tampering result of the image to be detected, further comprising: When the tampering result of the image to be detected is tampered, determine a target element in the fusion matrix whose value is greater than or equal to a preset confidence value of a pixel point, and determine a target pixel point in the image to be detected that has been tampered with based on the target element; Based on the target pixel point, the tampered area of ​​the image to be detected is displayed differentially.

7. The image tampering detection method according to claim 1, wherein the step of extracting text position features of the target text in the image to be detected comprises: Performing optical character recognition on the image to be detected to determine the text position of the target text in the image to be detected; Based on the text position of the target text in the image to be detected, a text position feature corresponding to the text position is determined.

8. The image tampering detection method according to claim 1, wherein obtaining the image to be detected comprises: Get the images to be detected uploaded by the user for approval from the server.

9. The image tampering detection method according to claim 2, wherein the step of determining the first image region of the target text in the image to be detected and the remaining second image region in the image to be detected based on the text position feature comprises: Input the text position feature and the image to be detected into a preset image information extraction module to determine a first image area of ​​the target text in the image to be detected and a remaining second image area in the image to be detected; The extracting the text color features and the background color features of the image to be detected based on the first image area and the second image area respectively includes: Based on the first image area and the second image area, the text color features and the background color features of the image to be detected are extracted by the preset image information extraction module.

10. The image tampering detection method according to claim 9, before inputting the text position feature and the image to be detected into a preset image information extraction module, further comprising: Performing data enhancement processing on the image to be detected to obtain the image to be detected after data enhancement.

11. An image tampering detection device, the device comprising: An image color feature extraction module, adapted to obtain an image to be detected, extract text position features of target text in the image to be detected, and extract image color features of the image to be detected based on the text position features; wherein the image color features include text color features and background color features; A discrete cosine transform feature extraction module, adapted to perform discrete cosine transform processing on the image to be detected, and extract discrete cosine transform features based on the result of the discrete cosine transform processing; A semantic segmentation module is suitable for determining an image color matrix corresponding to the image color feature based on the image color feature, and determining a discrete cosine transform matrix corresponding to the discrete cosine transform feature based on the discrete cosine transform feature; fusing the image color matrix with the discrete cosine transform matrix to obtain a fusion matrix corresponding to the image to be detected; wherein the value of each element in the fusion matrix represents the confidence value of the pixel point at the corresponding position in the image to be detected; and inputting the fusion matrix corresponding to the image to be detected into a preset image tampering classification module to determine the tampering result of the image to be detected.

12. A computer storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the method steps as claimed in any one of claims 1 to 10.

13. A computer program product storing at least one instruction, wherein the at least one instruction is loaded by a processor and executes the method steps according to any one of claims 1 to 10.

14. An electronic device comprising: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 10.

Images