Image tampering detection method, device and medium based on semantic understanding

Abstract

The present invention discloses an image tampering detection method, device and medium based on semantic understanding, which relates to the field of artificial intelligence. The method comprises: obtaining an image to be detected; extracting semantic information of the image to be detected through a semantic understanding layer; inputting the semantic information into a semantic segmentation layer, identifying each semantic category in the image to be detected, and obtaining a semantic segmentation result; inputting the semantic information into a tampering identification layer, extracting tampering features in the image to be detected, and obtaining a tampering identification result; if there are more than a set proportion of pixels in a semantic block belonging to the tampered area, the semantic block is determined as the final tampered area; during the model training process, the model parameters are updated by constraining the real tampered area in the sample image to approach the same semantic block in the predicted semantic segmentation result. The present invention obtains a tampering identification result that conforms to semantic understanding.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an image tampering detection method, device and medium based on semantic understanding. Background Art

[0002] With the rapid development of digital technology, image processing technology has been widely used, but it has also given rise to the problem of image tampering. Image tampering, that is, the unauthorized modification or forgery of images, has posed serious challenges to scientific research, news reporting and other fields.

[0003] Therefore, it is very important to provide an effective image tampering detection method to maintain the authenticity of the information. Summary of the invention

[0004] The embodiments of the present invention provide an image tampering detection method, device and medium based on semantic understanding to solve the above technical problems.

[0005] In a first aspect, an embodiment of the present invention provides an image tampering detection method based on semantic understanding, comprising:

[0006] Acquire the image to be detected;

[0007] Extracting semantic information of the image to be detected through the semantic understanding layer;

[0008] Inputting the semantic information into a semantic segmentation layer, identifying each semantic category in the image to be detected, and obtaining a semantic segmentation result, wherein the semantic segmentation result includes a plurality of semantic blocks with continuous areas, and pixels in the same semantic block have the same semantic category;

[0009] Inputting the semantic information into the tampering identification layer, extracting the tampering features in the image to be detected, and obtaining a tampering identification result, wherein the tampered area in the tampering identification result is 1, and the untampered area is 0;

[0010] If there are pixels in a semantic block with a set ratio or more belonging to the tampered area, the semantic block is determined as the final tampered area;

[0011] Among them, the semantic understanding layer, semantic segmentation layer and tampering identification layer constitute an image tampering detection model based on deep learning. During the model training process, the model parameters are updated by constraining the real tampering area in the sample image to approach the same semantic block in the predicted semantic segmentation results.

[0012] In a second aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0013] one or more processors;

[0014] a memory for storing one or more programs,

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the image tampering detection method based on semantic understanding described in any embodiment.

[0016] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image tampering detection method based on semantic understanding described in any embodiment.

[0017] In summary, the embodiment of the present invention provides an image tampering detection method based on semantic understanding, which analyzes the semantic features of the image, extracts the tampering features, and makes a comprehensive judgment on the tampering area to achieve higher accuracy. Specifically, the image is semantically understood by the image tampering detection model based on deep learning, and then semantic segmentation is performed to identify the semantic features in the image that are inconsistent with common sense; at the same time, the tampering features of the image are extracted by the tampering recognition layer, and then the image tampering recognition results and the image segmentation results are fused to obtain the final tampering area. In particular, during the model training process, the present invention updates the model parameters by constraining the real tampering area in the sample image to be segmented into a block with a continuous area (i.e., close to the same semantic block) in the predicted semantic segmentation results, so that the tampering area tends to be a complete semantic block, thereby obtaining a tampering recognition result that is more in line with semantic understanding. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a structural schematic diagram of an image tampering detection model based on semantic understanding provided by an embodiment of the present invention;

[0020] Figure 2 is a flow chart of an image tampering detection method based on semantic understanding provided by an embodiment of the present invention;

[0021] Figure 3 It is a structural schematic diagram of an image tampering detection model based on semantic understanding provided by an embodiment of the present invention;

[0022] Figure 4 is an image to be detected provided by an embodiment of the present invention;

[0023] Figure 5 The method of the embodiment is to Figure 4 After processing, the final tampered area is detected;

[0024] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0026] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0027] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0028] An embodiment of the present invention provides an image tampering detection method based on semantic understanding. In order to illustrate the method, a deep learning-based image tampering detection model that supports the implementation of the method is first introduced. Figure 1 is a structural diagram of an image tampering detection model based on semantic understanding provided by an embodiment of the present invention, such as Figure 1 As shown in FIG. 1 , the model includes a semantic understanding layer, a semantic segmentation layer, and a tampering identification layer. After the input image is input into the semantic understanding layer, the obtained results are respectively input into the semantic segmentation layer and the tampering identification layer. Among them, the semantic segmentation layer outputs the semantic segmentation result, and the tampering identification layer outputs the tampering identification result. The two results are fused to obtain the final tampering area of ​​the input image. The specific operation of each layer will be described in detail in the subsequent embodiments.

[0029] Based on the above model structure, Figure 2 is a flow chart of a method for detecting image tampering based on semantic understanding provided by an embodiment of the present invention. The method is executed by an electronic device, such as Figure 2 As shown, the specific steps include:

[0030] S110: Acquire an image to be detected.

[0031] This embodiment uses the image as a detection object and finds the tampered area in the image through subsequent operations.

[0032] S120 , extracting semantic information of the image to be detected through a semantic understanding layer.

[0033] The image to be detected is input into the semantic understanding layer in the image tampering detection model to extract semantic information in the image, and the semantic information is expressed as a feature map with the same size as the image to be detected.

[0034] In a specific embodiment, the semantic understanding layer includes multiple layers of semantic understanding units, and each layer of semantic understanding units includes a residual connection convolution module and a self-attention module. Figure 3 As shown in the figure, the number of layers of the semantic understanding unit is 5, and the convolution module in each layer of the semantic understanding unit uses a 3×3 convolution kernel with residual connection and the number of channels is 256.

[0035] Accordingly, the process of extracting semantic information through the semantic understanding layer may include the following steps:

[0036] Step 1: Input the image to be detected into the convolution module of the first-layer semantic understanding unit to extract local semantic information of the input image. The convolution operation operates each pixel with the pixels in the surrounding local area, which is conducive to extracting local semantic information in the image.

[0037] Step 2: After adding the extracted local semantic information to the input of the convolution module, continue to input the self-attention module of the first layer semantic understanding unit to learn global semantic information. The self-attention module can focus on the correlation between pixels in the global range, which is conducive to learning global semantic information.

[0038] Step three: input the global semantic information into the convolution module of the next-layer semantic understanding unit, and repeat the same operation as in the first-layer semantic understanding unit; then input the output of the next-layer semantic understanding unit into the next-layer semantic understanding unit, and repeat this cycle to gradually deepen the depth of the semantic information until the operations of all semantic understanding units are completed, and the semantic information feature map is obtained.

[0039] S130, inputting the semantic information into a semantic segmentation layer, identifying each semantic category in the image to be detected, and obtaining a semantic segmentation result, wherein the semantic segmentation result includes a plurality of semantic blocks with continuous areas, and pixels in the same semantic block have the same semantic category.

[0040] In this step, the semantic information feature map extracted in S120 is input into the semantic segmentation layer, and the semantic segmentation layer divides the feature map into a plurality of blocks of continuous area, and the pixels in each block have the same semantic category, and different semantic categories correspond to different label values. In this embodiment, these blocks are referred to as semantic blocks, and the semantic blocks together constitute a semantic segmentation result with the same size as the image to be detected. Specifically, the specific semantic categories in different images may be different. For example, in an image about architecture, the semantic categories may include buildings, people, frogs, etc., and in another image about scenery, the semantic categories may include sky, ocean, people, etc., but no matter what kind of semantic category, it can be distinguished by different label values.

[0041] In a specific embodiment, the semantic segmentation layer includes a convolution module with multiple layers of residual connections. And the convolution kernel scale of the convolution module in the semantic segmentation layer is larger than the convolution kernel scale of the convolution module in the semantic understanding layer, so as to improve the information receptive field and capture the block information in the image. For example, Figure 3 As shown in the figure, the number of convolution modules in the semantic segmentation layer is 7, and each convolution module uses a 7×7 convolution kernel with residual connection and 512 channels.

[0042] Accordingly, the process of obtaining the semantic segmentation result through the semantic segmentation layer may include the following steps:

[0043] Step 1: Input the semantic information into the first convolution module of the semantic segmentation layer to extract the deep features of the semantic information. In the deep features, the values ​​of each pixel in the blocks of the same semantic category will tend to be consistent, while the values ​​of blocks of different semantic categories will be different.

[0044] Step 2: After adding the deep features to the input of the first convolution module, continue to input the next convolution module and repeat the same operation as in the first convolution module; then input the output of the next convolution module into the next convolution module, and repeat this cycle to gradually improve the consistency of values ​​of the same semantic information block and the difference in values ​​of different blocks until the operations of all convolution modules are completed and the semantic segmentation result is obtained.

[0045] S140, inputting the semantic information into a tampering identification layer, extracting tampering features in the image to be detected, and obtaining a tampering identification result, wherein the tampered area in the tampering identification result is 1, and the untampered area is 0.

[0046] In this step, the semantic information feature map extracted in S120 is input into the tampering identification layer, and the tampering identification layer divides the pixels in the feature map into two areas, one is the tampered area (referred to as the tampered area), and the other is the untampered area (referred to as the untampered area). The pixel value in the tampered area is 1, and the pixel value in the untampered area is 0. The tampered part of the image is displayed in the form of a mask, and the mask image with the same size as the image to be detected is the tampering identification result.

[0047] In a specific embodiment, the tampering identification layer includes multiple layers of residual connected convolution modules, and the convolution kernel scale of the convolution module in the tampering identification layer is also larger than the convolution kernel scale of the convolution module in the semantic understanding layer, so as to improve the information receptive field and capture the block information in the image. Optionally, the number of layers, convolution kernel scale and number of channels of the convolution module in the tampering identification layer are respectively smaller than the number of layers, convolution kernel scale and number of channels of the convolution module in the semantic segmentation layer. This is because, compared with the tampering identification results, the information in the semantic segmentation results is more complex, and a relatively complex network should be used to improve the processing capabilities of complex information. For example, Figure 3 As shown in FIG. 1 , the number of convolution modules in the tampering identification layer is 3, and each convolution module uses a 5×5 convolution kernel with residual connection and has 256 channels.

[0048] Accordingly, the process of obtaining the tampering identification result by tampering the identification layer may include the following steps:

[0049] Step 1: Input the semantic information into the first convolution module of the tampering identification layer to extract another deep feature of the semantic information. In the deep feature, the values ​​of the tampered area and the untampered area are different, but the pixel values ​​inside the area tend to be consistent.

[0050] Step 2: After adding the depth feature to the input of the first convolution module, continue to input the next convolution module and repeat the same operation as in the first convolution block; then input the output of the next convolution module into the next convolution module, and repeat this cycle to gradually improve the consistency of values ​​within the tampered area and the untampered area until the operations of all convolution modules are completed and the tampering identification result is obtained.

[0051] S150: If pixels in a semantic block that have a set ratio or more belong to the tampered area, determine the semantic block as the final tampered area.

[0052] This step realizes the fusion of the semantic segmentation result and the tampering identification result, and obtains the tampering identification result guided by the semantic segmentation. Specifically, in the semantic segmentation result obtained in S130, if there is a semantic block with a set proportion (such as 70%) of the area, which is identified as a tampered area in the tampering identification result obtained in S140, then the semantic block is set as the final tampering area as a whole, that is, the final tampering area in this embodiment is a complete semantic block, which maintains the semantic integrity of the tampering area.

[0053] In order to achieve the above detection effect, this embodiment provides a training method for the above image tampering detection model, by constraining the real tampering area in the sample image to be segmented into a block with a continuous area (i.e., close to the same semantic block) in the predicted semantic segmentation result, to update the model parameters. In a specific implementation, the training process specifically includes the following steps:

[0054] Step 1: Build a sample image library, and annotate the true semantic segmentation results and tampering identification results of each sample image. Optionally, in the annotation of the semantic segmentation results, pixels of the same semantic category are divided into a block, and all pixels in the block are annotated with the label value corresponding to the semantic category; different semantic categories correspond to different label values. Optionally, in the annotation of the tampering identification results, all pixels in the tampered area are annotated as 1, and all pixels in the untampered area are annotated as 0.

[0055] Step 2: Input the sample image into the semantic understanding layer of the image tampering detection model, and the semantic segmentation layer and the tampering identification layer predict the semantic segmentation result and the tampering identification result respectively. For the convenience of distinction and description, this embodiment records the predicted semantic segmentation result as S1 and the true semantic segmentation result as S2; the predicted tampering identification result as N1 and the true tampering identification result as N2.

[0056] Step 3: Delete the corresponding pixels in the predicted semantic segmentation result (S1) of the real untampered area in the sample image (the area with a value of 0 in N2) to obtain a semantic segmentation result that only retains the tampered area, which is recorded as H(S1).

[0057] Step 4: Update the model parameters by minimizing the difference between the predicted semantic segmentation result and the true semantic segmentation result, minimizing the difference between the predicted tampering identification result and the true tampering identification result, and minimizing the sum of squared deviations from the mean of the semantic segmentation result that only retains the tampered area. The minimization of the sum of squared deviations from the mean is used to constrain the true tampered area to approach the same semantic block in the predicted semantic segmentation result.

[0058] Optionally, you can construct the following loss function and update the model parameters by minimizing the loss function:

[0059] Lossfunction=α×E(S1,S2)+β×E(N1,N2)+γ×STD(H(S1))

[0060] Among them, Lossfunction represents the loss function value, E(S1, S2) represents the difference between the predicted semantic segmentation result and the true semantic segmentation result. For example, the Euclidean distance between S1 and S2 can be calculated; E(N1, N2) represents the difference between the predicted tampering recognition result and the true tampering recognition result. Similarly, the Euclidean distance between N1 and N2 can be calculated; STD(H(S1)) represents the sum of squares of the mean deviation of the semantic segmentation result that only retains the tampered area. The calculation formula of STD(H(S1)) is:

[0061]

[0062] Among them, x i represents the semantic label value of the i-th pixel in H(S1), n ​​represents the number of pixels in H(S1), and μ represents the average value of the semantic label values ​​of all pixels in H(S1). It is worth emphasizing that only the pixels of the tampered area in S1 are retained in H(S1), and the rest of the pixels are deleted. Therefore, the number of pixels in H(S1) is less than the number of pixels in S1. By minimizing STD(H(S1)), the semantic labels of the tampered areas in S1 can be made consistent, that is, this embodiment hopes that the tampered area will be segmented into a whole area in the semantic segmentation step.

[0063] Furthermore, α, β and γ are three weights respectively. Since the semantic understanding ability is more complex and important, α=γ>β can be set first for training to enable the model to have semantic understanding ability; then α=β=γ can be set for secondary training to enable the network with semantic understanding ability to obtain tampering detection ability. In a specific embodiment, the first 50% of the rounds can be set to α=1, β=0.5, β=1, and the last 50% of the rounds can be set to α=1, β=1, γ=1, so that the network model has the ability of semantic understanding and then obtains the ability of tampering detection.

[0064] The following is a specific example to demonstrate the detection effect of the method of this embodiment. Figure 4 This is an image to be detected. It can be seen that there is a frog in the lower right corner of the image. From the perspective of semantic understanding, there should not be such a big frog of this color here. Figure 5 This is the tampered area finally detected by the method of the embodiment. It can be seen that the method of this embodiment can identify the tampered area in the image from the perspective of semantic understanding and maintain a fairly high accuracy.

[0065] In summary, most of the existing image tampering detection methods identify tampering areas by extracting tampering trace features. The detailed features concerned include texture, shape, etc., but ignore the understanding of the image semantics itself. The tampering area in the image often violates the consistency of the natural semantics of the image. Therefore, this embodiment provides an image tampering detection method based on semantic understanding, which analyzes the semantic features of the image and extracts the tampering features to make a comprehensive judgment on the tampering area to achieve higher accuracy. Specifically, this embodiment uses an image tampering detection model based on deep learning to understand the image semantically and then perform semantic segmentation to identify the semantic features that are not in line with common sense in the image; at the same time, the tampering features of the image are extracted through the tampering recognition layer, and then the image tampering recognition results and the image segmentation results are merged to obtain the final tampering area. In particular, during the model training process, this embodiment constrains the real tampering area in the sample image to approach the same semantic block (i.e., belong to the same semantic block) in the predicted semantic segmentation results to update the model parameters so that the tampering area tends to a complete semantic block, thereby obtaining a tampering recognition result that is more in line with semantic understanding.

[0066] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more. Figure 6 A processor 60 is taken as an example; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected by a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0067] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the image tampering detection method based on semantic understanding in the embodiment of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 61, that is, realizing the above-mentioned image tampering detection method based on semantic understanding.

[0068] The memory 61 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 61 may further include a memory remotely arranged relative to the processor 60, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0069] The input device 62 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 63 may include a display device such as a display screen.

[0070] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the image tampering detection method based on semantic understanding of any embodiment is implemented.

[0071] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0072] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0073] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0074] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting image tampering based on semantic understanding, characterized in that: include: Acquire the image to be detected; Extracting semantic information of the image to be detected through the semantic understanding layer; Inputting the semantic information into a semantic segmentation layer, identifying each semantic category in the image to be detected, and obtaining a semantic segmentation result, wherein the semantic segmentation result includes a plurality of semantic blocks with continuous areas, and pixels in the same semantic block have the same semantic category; Inputting the semantic information into the tampering identification layer, extracting the tampering features in the image to be detected, and obtaining a tampering identification result, wherein the tampered area in the tampering identification result is 1, and the untampered area is 0; If there are pixels in a semantic block with a set ratio or more belonging to the tampered area, the semantic block is determined as the final tampered area; The semantic understanding layer, semantic segmentation layer and tampering identification layer constitute an image tampering detection model based on deep learning. During the model training process, the model parameters are updated by constraining the real tampering area in the sample image to be close to the same semantic block in the predicted semantic segmentation results. The updating of the model parameters by constraining the real tampered area in the sample image to be close to the same semantic block in the predicted semantic segmentation result includes: Inputting the sample image into the semantic understanding layer of the image tampering detection model, and respectively predicting the semantic segmentation result and the tampering identification result by the semantic segmentation layer and the tampering identification layer; Deleting pixels corresponding to the real untampered area of ​​the sample image in the predicted semantic segmentation result, so as to obtain a semantic segmentation result retaining only the tampered area; The model parameters are updated by minimizing the difference between the predicted semantic segmentation result and the true semantic segmentation result, minimizing the difference between the predicted tampering identification result and the true tampering identification result, and minimizing the sum of squared deviations from the mean of the semantic segmentation result retaining only the tampered area, wherein the minimization of the sum of squared deviations from the mean is used to constrain the true tampered area to approach the same semantic block in the predicted semantic segmentation result.

2. The method according to claim 1, characterized in that The model parameters are updated by minimizing the difference between the predicted semantic segmentation result and the true semantic segmentation result, minimizing the difference between the predicted tampering identification result and the true tampering identification result, and minimizing the sum of squares of the mean deviation of the semantic segmentation result retaining only the tampered area, including: The model parameters are updated by minimizing the following loss function: Lossfunction=α×E(S1,S2)+β×E(N1,N2)+γ×STD(H(S1)) Among them, Lossfunction represents the loss function value, E(S1, S2) represents the difference between the predicted semantic segmentation result and the true semantic segmentation result, E(N1, N2) represents the difference between the predicted tampering recognition result and the true tampering recognition result, STD(H(S1)) represents the sum of squared deviations from the mean of the semantic segmentation results that only retain the tampered area, and α, β, and γ are three weights respectively.

3. The method according to claim 2, characterized in that The model parameters are updated by minimizing the following loss function, including: Set α=γ>β for training to make the model have semantic understanding ability; Setting α=β=γ for secondary training enables the network with semantic understanding ability to acquire tampering detection ability.

4. The method according to claim 1, characterized in that: The semantic understanding layer includes multiple layers of semantic understanding units, each layer of semantic understanding units includes a residual connected convolution module and a self-attention module; The step of extracting semantic information of the image to be detected through the semantic understanding layer includes: Inputting the image to be detected into the convolution module of the first-layer semantic understanding unit to extract local semantic information of the image to be detected; After adding the extracted local semantic information to the input of the convolution module, the information is input into the self-attention module of the first-layer semantic understanding unit to learn the global semantic information; The global semantic information is input into the convolution module of the next layer of semantic understanding units to perform the same operation; this cycle is repeated to gradually deepen the depth of the semantic information until the operations of all semantic understanding units are completed.

5. The method according to claim 1, characterized in that The semantic segmentation layer includes a convolutional module with multiple layers of residual connections; The step of inputting the semantic information into a semantic segmentation layer, identifying each semantic category in the image to be detected, and obtaining a semantic segmentation result includes: Inputting the semantic information into the first convolution module of the semantic segmentation layer, extracting the deep features of the semantic information, and making the values ​​of blocks of the same semantic category tend to be consistent; After adding the depth feature to the input of the first convolution module, the next convolution module is input to perform the same operation; this cycle is repeated to gradually improve the consistency of values ​​of blocks of the same semantic category until the operations of all convolution modules are completed.

6. The method according to claim 1, characterized in that The tampering identification layer includes a convolutional module with multiple layers of residual connections; The step of inputting the semantic information into a tampering identification layer, extracting tampering features in the image to be detected, and obtaining a tampering identification result includes: Inputting the semantic information into the first convolution module of the tampering identification layer, extracting the deep features of the semantic information, and making the values ​​inside the tampered area and the untampered area tend to be consistent; After adding the depth feature to the input of the first convolution module, continue to input the next convolution module to perform the same operation; this cycle is repeated to gradually improve the consistency of values ​​within the tampered area and the untampered area until the operations of all convolution modules are completed.

7. The method according to claim 1, characterized in that The semantic understanding layer includes multiple layers of semantic understanding units, each layer of semantic understanding units includes a residual connected convolution module and a self-attention module; the semantic segmentation layer and the tampering identification layer both include multiple layers of residual connected convolution modules; The convolution kernel scales of the convolution modules in the semantic segmentation layer and the tampering identification layer are larger than the convolution kernel scale of the convolution module in the semantic understanding layer, so as to improve the ability to capture block information; The number of layers, convolution kernel scale and number of channels of the convolution module in the semantic segmentation layer are respectively greater than the number of layers, convolution kernel scale and number of channels of the convolution module in the tampering identification layer, so as to improve the processing capability of complex information.

8. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the image tampering detection method based on semantic understanding as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the image tampering detection method based on semantic understanding as described in any one of claims 1-7 is implemented.

Images