Image forgery detection method and device based on size grading, equipment and medium

Abstract

The invention relates to the technical field of image recognition and financial science and technology, and discloses an image forgery detection method, device and equipment based on granularity grading and a medium, and the method comprises the steps: determining a detection level of a pre-obtained target image according to a user demand, and recognizing a to-be-detected level image corresponding to the target image according to the detection level; determining a detection branch according to the detection hierarchy, and mapping the to-be-detected hierarchy image and the detection branch to obtain an image branch mapping relationship; inputting the to-be-detected hierarchical images into the detection branches one by one through an image branch mapping relation, and outputting tampering masks of the to-be-detected hierarchical images; fusing the tampering mask and the target image to obtain a fused image, and positioning the position of a forged image in the target image according to the fused image; and determining a forgery behavior in the target image through the position of the forgery image. According to the invention, the accuracy of image forgery detection can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition and financial technology, and in particular to a method, device, equipment and medium for detecting image forgery based on granularity classification. Background Art

[0002] With the continuous advancement of science and technology and the vigorous development of electronic technology, various image tampering methods have emerged one after another. For example, in the insurance business, the monitoring of people and vehicles is often involved. In order to protect the life and property safety of the insured persons, as well as to protect the authenticity of the image, maintain credibility and fairness, prevent the spread of false information, and reduce fraud and infringement, it is necessary to determine whether the image has been tampered with and locate the suspicious location of the image tampering to identify the forgery in the image.

[0003] Existing image forgery detection technology can determine whether any image has been tampered with and roughly locate the suspected location of image tampering. However, when there is complex lighting, occlusion, deformation or multi-layer synthesis in the image, it is impossible to accurately judge the authenticity of the image, and there may be false positives and missed negatives. In addition, there are multiple different forgery traces or signs in the image, which increases the complexity of image forgery detection, resulting in low accuracy when performing image forgery detection. Summary of the Invention

[0004] The present invention provides an image forgery detection method, device, equipment and medium based on granularity classification, so as to solve the technical problem of low accuracy in image forgery detection.

[0005] In a first aspect, a method for detecting image forgery based on granularity classification is provided, comprising:

[0006] Determine the detection level of the pre-acquired target image according to preset user requirements, and identify the image of the to-be-detected level corresponding to the target image according to the detection level;

[0007] Determine a detection branch according to the detection level, and map the image of the level to be detected with the detection branch to obtain an image branch mapping relationship;

[0008] Inputting the to-be-detected hierarchical image into the detection branch one by one through the image branch mapping relationship, and outputting the tampering mask of the to-be-detected hierarchical image;

[0009] fusing the tampering mask and the target image to obtain a fused image, and locating the forged image position in the target image according to the fused image;

[0010] The forgery behavior in the target image is determined according to the forged image position.

[0011] In a second aspect, a device for detecting image forgery based on granularity classification is provided, comprising:

[0012] The to-be-detected level image recognition module is used to determine the detection level of the pre-acquired target image according to preset user requirements, and to identify the to-be-detected level image corresponding to the target image according to the detection level;

[0013] An image branch mapping relationship determination module is used to determine a detection branch according to the detection level, and map the image of the level to be detected with the detection branch to obtain an image branch mapping relationship;

[0014] a tampering mask output module, configured to input the to-be-detected hierarchical image into the detection branch one by one through the image branch mapping relationship, and output the tampering mask of the to-be-detected hierarchical image;

[0015] a forged image position locating module, configured to fuse the tampering mask and the target image to obtain a fused image, and locate the forged image position in the target image according to the fused image;

[0016] The forgery behavior determination module is configured to determine the forgery behavior in the target image according to the forged image position.

[0017] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned granularity-based image forgery detection method are implemented.

[0018] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned image forgery detection method based on granularity classification are implemented.

[0019] The embodiments of the present invention can optimize detection strategies for specific scenarios and targets based on preset user needs, thereby improving the pertinence and accuracy of detection. Image forgery detection is performed hierarchically, with forged images divided into multiple scales based on requirements. Detection is performed on images within a specific scale, enabling detailed identification of areas where forgery or tampering occurs. The detected mask contains multi-scale information about the image to be detected, avoiding the problems of information loss and gradient vanishing during network transmission. The output results are more accurate, thus avoiding the drawback of performance degradation in complex backgrounds and significantly improving accuracy. Therefore, the image forgery detection method, device, equipment, and medium based on granularity grading proposed by the present invention can solve the problem of accuracy in image forgery detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 2. It is a schematic diagram of an application environment of an image forgery detection method based on granularity classification according to an embodiment of the present invention;

[0022] Figure 2 This is a flow chart of a method for detecting image forgery based on granularity classification according to an embodiment of the present invention;

[0023] Figure 3 yes Figure 1 A schematic flow chart of a specific implementation of step S1;

[0024] Figure 4 yes Figure 1 A schematic flow chart of a specific implementation of step S2;

[0025] Figure 5 is a structural diagram of an image forgery detection device based on granularity classification in one embodiment of the present invention;

[0026] Figure 6 is a structural diagram of a computer device in one embodiment of the present invention;

[0027] Figure 7 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort shall fall within the scope of protection of the present invention.

[0029] The image forgery detection method based on granularity classification provided by the embodiment of the present invention can be applied in the following fields: Figure 1In an application environment, the client communicates with the server through a network. The server can determine the detection level of the pre-acquired target image according to the preset user requirements through the client, identify the level image to be detected corresponding to the target image according to the detection level; determine the detection branch according to the detection level, map the level image to be detected with the detection branch, and obtain an image branch mapping relationship; input the level image to be detected into the detection branch one by one through the image branch mapping relationship, and output the tampering mask of the level image to be detected; fuse the tampering mask and the target image respectively to obtain a fused image, locate the forged image position in the target image according to the fused image; determine the forged behavior in the target image through the forged image position , and feed back the forgery behavior to the client. In the present invention, according to the preset user needs, the detection strategy can be optimized for specific scenarios and targets to improve the pertinence and accuracy of the detection; the image forgery detection is carried out in a hierarchical manner, and the forged pictures are divided into multiple scales according to the needs. The images within a specific scale are detected, and it is possible to identify in detail which areas have forgery or tampering behaviors; the mask after detection contains multi-scale information of the image to be detected, which avoids the problems of information loss and gradient disappearance in network transmission, and the output result is more accurate, thereby avoiding the disadvantage of performance degradation in complex background conditions, and greatly improving its accuracy. Among them, the client can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0030] See also Figure 2 As shown, Figure 2 A flowchart of a granularity-based image forgery detection method provided in an embodiment of the present invention includes the following steps:

[0031] S1. Determine a detection level of a pre-acquired target image according to preset user requirements, and identify a level image to be detected corresponding to the target image according to the detection level.

[0032] In an embodiment of the present invention, the user demand refers to the image demand that the user needs to detect in combination with business needs, that is, the user needs to focus on detecting whether one or several parts in the picture have been tampered with or forged, such as the demand for detection of images of people, vehicles, etc. in related insurance business; the detection level refers to the detection branch of the detection, which is divided into three levels: local, target and global according to different user needs. Each level focuses on different aspects of the image to meet specific analysis needs; the pre-acquired target image refers to the image data collected based on user needs. For example, if the user demand is to collect person images, the target image is the person image; if the user demand is to collect vehicle images, the target image is the vehicle image.

[0033] In an embodiment of the present invention, determining the detection level of the pre-acquired target image according to a preset user requirement includes:

[0034] Extracting a demand hierarchy from the user demand;

[0035] Dividing the requirement hierarchy into local image requirements, target image requirements and global image requirements;

[0036] Determining, according to the local image requirement, a detection level of the target image as a local detection level;

[0037] Determining, according to the target image requirement, a detection level of the target image as a target detection level;

[0038] According to the global image requirement, the detection level of the target image is determined to be a global detection level.

[0039] In detail, the level of target image detection is determined based on user needs. For example, if the user needs to detect a local area, the level of demand is local image demand; if the user needs to detect a target area, the level of demand is target image demand; if the user needs to detect a global area, the level of demand is global image demand. Local image demand means that the user wants to focus on a local area in the image; target image demand means that the user wants to focus on a specific object or target in the image; global image demand means that the user wants to focus on the content of the entire image.

[0040] Specifically, the detection level of the target image is determined as the local detection level according to the local image requirements, the detection level of the target image is determined as the target detection level according to the target image requirements, and the detection level of the target image is determined as the global detection level according to the global image requirements. The local detection level mainly focuses on the details of the target image, the target detection level mainly identifies and locates specific objects in the target image, and the global detection level mainly analyzes and understands the content of the entire image.

[0041] Furthermore, the images to be detected are determined according to different detection levels, and then the images to be detected determined according to different detection levels are input into different level areas to achieve detection from local to global, thereby ensuring the accuracy of image detection.

[0042] In the embodiment of the present invention, the image at the level to be detected refers to an image obtained by processing the target image based on the image requirements corresponding to the detection level, and the processed image is used as the image to be detected corresponding to each level of image forgery detection.

[0043] In the embodiment of the present invention, referring to Figure 3As shown, the step of identifying the image at the level to be detected corresponding to the target image according to the detection level includes:

[0044] S31. When the detection level is a local detection level, locally crop the target image according to the local image requirement to obtain a local image to be detected corresponding to the local detection level;

[0045] S32. When the detection level is a target detection level, extracting a target detection image corresponding to the target image according to the target image requirement, and determining the target detection image as a local target to-be-detected image corresponding to the target detection level;

[0046] S33: When the detection level is a global detection level, determine the target image as a global image to be detected corresponding to the global detection level.

[0047] In detail, the local detection layer focuses on a specific small area in the image, crops the target image according to the image area that needs to be obtained in the local image requirements, and uses the cropped image as the local image to be detected, which only contains the required local information; the target detection layer aims to identify specific objects in the image, and uses the target detection algorithm (such as YOLO, etc.) to process the target image, identify and locate specific objects in the image, and the generated target detection map is an image that is processed and focuses on specific objects; the global detection layer processes the entire image, focuses on overall features, does not target specific areas, but takes the entire target image as input for comprehensive analysis. The entire target image is called the global image to be detected.

[0048] For example, if the target image is an image of a vehicle, a person, and a background, the local image to be detected is the person's face and the vehicle's license plate; the target image to be detected is the entire person's body and the vehicle body; and the global image to be detected is the entire target image.

[0049] Furthermore, the detection branch is determined based on the detection level, so that the image to be detected is input into three detection branches in different hierarchical areas. The detection sizes of the three branches are gradually reduced, and detection is performed simultaneously to achieve detection from global to local.

[0050] S2. Determine a detection branch according to the detection level, map the image of the level to be detected with the detection branch, and obtain an image branch mapping relationship.

[0051] In an embodiment of the present invention, the detection branch refers to mask detection of the image to be detected involved in different levels of areas. Each branch will output the detected tampering mask. The area detected at each level is different. Each level will output a predicted Mask (tampering mask). Mask represents the result of forgery detection at each level. Finally, the abnormal behavior to which the image belongs is determined based on the result, and the abnormal behavior is classified.

[0052] In detail, when the detection level is the local detection level, the corresponding detection branch is local detection; when the detection level is the target detection level, the corresponding detection branch is target detection; when the detection level is the global detection level, the corresponding detection branch is background detection.

[0053] Furthermore, the image to be detected is input into three detection branches in different hierarchical areas. The detection sizes of the three branches are gradually reduced, but are performed simultaneously. Therefore, the image to be detected needs to be input into the corresponding detection branch to obtain the tampering mask detected at each level.

[0054] In an embodiment of the present invention, the branch mapping relationship refers to the correspondence and relationship between the hierarchical image to be detected and different detection branches, which clarifies which detection branch each hierarchical image should be processed by. Each hierarchical image corresponds one-to-one to a specific detection branch (local detection, target detection, global detection) according to its characteristics. Through the mapping relationship, it can be guided to the most suitable detection branch according to the characteristics of the image, thereby improving processing efficiency and accuracy.

[0055] In the embodiment of the present invention, referring to Figure 4 As shown, mapping the to-be-detected hierarchical image with the detection branch to obtain an image branch mapping relationship includes:

[0056] S41, dividing the detection branch into a local detection branch, a target detection branch and a global detection branch;

[0057] S42, making one-to-one correspondence between the hierarchical images in the hierarchical images to be detected and the local detection branch, the target detection branch, and the global detection branch respectively;

[0058] S43. Determine the relationship between the corresponding hierarchical image and the detection branch as an image branch mapping relationship.

[0059] In detail, the detection branches are divided into local detection branches, target detection branches and global detection branches according to different detection tasks. The local detection branches focus on specific small areas of the image; the target detection branches are used to identify and locate specific objects in the image; the global detection branches process the entire image, and then the detection level images are matched one by one with each detection branch. The local images to be detected correspond to the local detection branches, the local target images to be detected correspond to the target detection branches, and the global images to be detected correspond to the global detection branches, thus establishing a clear relationship between the images at each level and the corresponding detection branches.

[0060] Exemplarily, if the target image is a partial image to be detected, the face of the person and the license plate of the vehicle are detected. In the local detection branch, whether the face of the person is real, whether there is any forgery operation, whether there is any face-changing technology operation and other types of forgery operations, and at the same time, license plate detection is realized to check whether the numbers and letters on the license plate have been tampered with. If so, a mask predicting face forgery and license plate forgery is output; if the target image is a target image to be detected, the entire body of the person and the vehicle are detected. In the target detection branch, it is judged whether the person and the vehicle as a whole have been replaced or tampered with. The judgment standard is PS traces, including but not limited to collage (cutting out the object of Figure A and pasting it on the background of Figure B, which often has blurred edges and jagged edges), erasure (erasing an object somewhere in the picture, which often has irregular textures and blurs). If the target image is a global image to be detected, the entire target image of the person is detected, and tampering in the background is detected to detect whether the background scene is real.

[0061] Furthermore, in order to detect the forgery results in the target image, it is necessary to input the hierarchical image to be detected into the corresponding detection branch one by one, output the tampering mask of the hierarchical image to be detected, and then obtain the forgery detection results of all areas by fusing the masks of the three branches with the original image.

[0062] S3. Input the to-be-detected hierarchical image into the detection branch one by one through the image branch mapping relationship, and output the tampering mask of the to-be-detected hierarchical image.

[0063] In an embodiment of the present invention, Mask RCNN (Mask Region-based Convolutional Neural Network, a deep learning model for target detection and instance segmentation) is used in the detection branch to complete the detection task, and the 80-category pre-trained model of the MS COCO (Microsoft Common Objects in Context, target detection) dataset is used to complete the detection task. If other suspicious parts need to be detected in the business, the detection task can be completed by adding labels with a small amount of annotations (about 500 pictures).

[0064] In the embodiment of the present invention, the tampering mask refers to a binary image of a suspicious position in the detected target image, and the value range of the tampering mask is 0-1.

[0065] In the embodiment of the present invention, the step of inputting the to-be-detected hierarchical image into the detection branch one by one through the image branch mapping relationship and outputting the tampering mask of the to-be-detected hierarchical image includes:

[0066] Extracting a hierarchical detection branch corresponding to the hierarchical image to be detected according to the image branch mapping relationship, and extracting multi-scale image features of the hierarchical image to be detected using a feature extraction model in the hierarchical detection branch;

[0067] An image feature corresponding to the minimum scale in the multi-scale image features is used as a target scale image feature, an upsampling operation is performed on the target scale image feature, and a target mask feature corresponding to the upsampled target scale image feature is output through a preset convolution head;

[0068] The target mask feature is fused with the preset initial mask feature to obtain a mask fusion feature, and the mask fusion feature is supervised by a preset true label to obtain a target-level tampering mask;

[0069] The target-level tampering mask is updated to the initial mask feature, the image feature of the next scale corresponding to the minimum scale is updated to the image feature corresponding to the minimum scale, and the process returns to the step of using the image feature corresponding to the minimum scale in the multi-scale image features as the target-scale image feature, until the image features corresponding to all scales in the multi-scale image features are used as the target-scale image feature, and the tampering mask corresponding to the image of the level to be detected is output.

[0070] In detail, the UNet (U-shapedNetwork, deep learning network model) network structure is used to extract multi-scale image features of the hierarchical image to be detected, so as to maintain the high resolution of the input and retain small-sized tampering traces well. The multi-scale image features refer to images of different scales corresponding to the image to be detected. The UNet network is divided into two parts: Encoder (encoding layer, gradually deepening and reducing the resolution) and Decoder (decoding layer, restoring the original resolution). Its structural feature is to directly cascade the high-resolution features of the low layer to the encoding part, and test the more local tampering operations such as erasure. The tampering features are relatively subtle and do not have the structural features of the object (for example, the collage of characters makes the entire character forged. At this time, the object semantic information encoded by the deep features is more effective; while smearing is mostly point operation, which has no semantic features, and the low-level features are more discriminative). It can retain traces well, and the feature layers of multiple scales can better adapt to tampering sizes of different sizes.

[0071] Specifically, based on the original UNet network, after upsampling the image features with the smallest scale, the mask Mask1 of the corresponding size is output through the Head (convolution head) in the first layer. Mask1 is then upsampled and fused into the next Head to help the next network layer output a new prediction Mask2. And so on, after passing through the last layer of the network, the final prediction Mask4 is output. The final model output result contains features of other scales and levels, and in the Mask4 prediction process, the prediction result of Mask3 is directly integrated. At the same time, the real image label (groundtruth) of the target image supervises the Masks of the four scales, that is, the small-scale prediction also has a supervisory adjustment effect on the features of modules 1, 2, and 3. Therefore, it can be expressed as 4 The Mask output by the model is the model prediction result, which is deconvolved to 2 times the size and multiplied with the feature predicted by the Head of the next layer to obtain the fused feature. The Head refers to the output part of the model, which is responsible for converting the feature-extracted data into the final output format. In UNet, the Head usually includes the last few layers of the network to generate masks or classification results, including convolution layers, activation functions and output size adjustment. The convolution layer is used to map the feature map to the number of target categories, the activation function is used to generate probability distribution, and the output size adjustment ensures that the output matches the size of the input image. The initial mask feature is the tampering mask generated by the first layer itself. First, the tampering mask generated by the first layer is intersected with itself, and then the initial mask feature is updated.

[0072] Furthermore, in order to avoid information loss and gradient disappearance problems and make the results more accurate, the real image label (groundtruth) is used to supervise the generation of Masks of four scales, and the Loss function is used to supervise the generation of Masks. The loss function is composed of three functions, namely BCE (cross entropy loss function), Dice (similarity coefficient loss function), and Lovasz (expansion loss function), among which BCE (cross entropy loss function) L1 is Where Loss is the cross entropy loss value, n is the number of categories, here there are only two categories: real-0 and tampered-1, y i is the label value of the i-th sample. The true category of sample i is equal to c and takes 1, otherwise it takes 0, y i ' is the probability that the observed sample i belongs to category c; Dice (similarity coefficient loss function) L2 is Among them, DiceLoss represents the similarity loss value, | X∩Y | represents the intersection between X and Y, | X| and | Y represents the number of elements in X and Y respectively. The coefficient 2 in the numerator is because the denominator repeatedly counts the common elements between X and Y. X represents the real label image and Y represents the predicted image; Lovasz (expanded loss function) L3 is where y * is the true label image, that is, the real pixel -0, or the tampered pixel -1, To predict the label, the predicted value has been binarized to 0 or 1 according to 0.5, M c y * and The number of elements that do not belong to the same label; c is the category. When c is 1, it means the pixel is tampered. The definition of Lovasz loss is the union of the number of elements whose predictions are inconsistent with the labels (GT is the tampered pixel) and the number of pixels whose predictions are inconsistent, where L1:L2:L3=1.0:0.5:0.3.

[0073] Furthermore, based on the different areas detected at each level, each level will output a Mask, which represents the result of forgery detection at this level. In order to obtain the forgery detection results of all areas after the Masks of the three branches are fused with the original image, it is necessary to fuse the tampering mask output by each level with the original image to obtain the forgery detection results of all areas.

[0074] S4. Fusing the tampering mask and the target image to obtain a fused image, and locating the forged image position in the target image according to the fused image.

[0075] In the embodiment of the present invention, the fused image refers to the process of synthesizing an image by combining the generated tampering mask with the original target image, which aims to preserve and highlight important information or features in different images.

[0076] In an embodiment of the present invention, fusing the tampering mask and the target image to obtain a fused image includes:

[0077] determining a hierarchical tampering image of the hierarchical image to be detected according to the tampering mask;

[0078] Converting the target image into a target binary image;

[0079] An intersection operation is performed on the hierarchical tampered image and the target binary image to obtain a fused image.

[0080] In detail, according to the tampering mask output by each detection branch, the corresponding hierarchical tampering image is extracted from the image to be detected, the image to be detected is multiplied by the tampering mask to retain the marked area, the original target image is binarized, and the pixels in the image are divided into foreground (target) and background (non-target) using threshold segmentation, where the foreground pixels are 1 and the background pixels are 0, and the hierarchical tampering image and the target binary image are subjected to pixel-by-pixel intersection operation, that is, the suspicious position Mask (denoted as M1) obtained by the forgery detection is directly intersected with the detected area Mask (denoted as M0, M0 is a value of 0-1), that is, M=M0*M1, to obtain a fused image.

[0081] Specifically, the position of the forged image in the target image is located based on the fused image, that is, the fused image is binarized to more clearly separate the potential forged areas, and connected component analysis is performed on the binarized fused image to identify and mark all forged areas, so that the forged areas are determined as the positions of the forged images in the target image.

[0082] Furthermore, in order to identify forgery in the target image, it is necessary to determine abnormal behavior of the target image based on the forged image position and classify the abnormal behavior to determine the forgery in the target image.

[0083] S5. Determine the forgery behavior in the target image according to the forged image position.

[0084] In an embodiment of the present invention, the forgery behavior refers to tampering with, modifying or synthesizing the target image, changing the content of the original image by adding, deleting or modifying elements in the image, and synthesizing multiple images or elements into a new image to create a false scene or information, etc., such as detecting whether a person's face has been changed by face-changing technology or whether the numbers and letters on the license plate have been tampered with.

[0085] In the embodiment of the present invention, determining the forgery behavior in the target image by using the forged image position includes:

[0086] identifying a forged image in the target image according to the forged image position;

[0087] determining a forgery type in the target image based on the forged image;

[0088] The forgery behavior in the target image is determined according to the forgery type.

[0089] Specifically, based on the size, shape and other characteristics of the area, noise and non-forged areas are filtered out, and the detected forged areas are marked in the target image, for example, by highlighting them with a frame or color. Finally, an output image containing the marked forged areas is generated. Once the forged areas are identified, the specific type of forgery needs to be determined. The forgery types include but are not limited to collage (cutting out the object in image A and pasting it on the background of image B, which often results in blurred edges and jagged edges) and erasure (erasing an object somewhere in the image, which often results in irregular textures and blur).

[0090] Specifically, the forgery behavior in the target image can be determined according to the forgery type. When the forgery type is a collage type, the forgery behavior is to cut out the object in picture A and paste it on the background of picture B, which often results in blurred edges and jagged edges; when the forgery type is an erasure type, the forgery behavior is to erase an object somewhere in the picture, which often results in irregular textures and blur.

[0091] Furthermore, ①, ②, and ③ are used to represent different levels set according to user needs, namely ① local detection, ② target detection, and ③ background detection. The area detected at each level is different, and each level will output a predicted Mask. This Mask represents the result of the forgery detection at this level. Finally, the abnormal behavior to which it belongs is determined based on the result, and the abnormal behavior is classified. The photo of the person and the car to be detected is input into the three-level branches. In level ①, whether the face of the person is real is detected; level ② person detection is a forgery detection of the entire human body and vehicle body of the target, and it is determined whether the person and the vehicle as a whole have been replaced or tampered with; level ③ branch detects tampering in the background and detects whether the background scene is real. The masks of the three branches are fused with the original image to obtain the forgery detection results of all areas, and the abnormal behavior is classified according to the results.

[0092] It can be seen that in the above scheme, according to the preset user needs, the detection strategy can be optimized for specific scenarios and targets to improve the pertinence and accuracy of detection; the image forgery detection is carried out in a hierarchical manner, the forged images are divided into multiple scales according to the needs, and the images within a specific scale are detected, which can identify in detail which areas have forgery or tampering behavior; the mask after detection contains multi-scale information of the image to be detected, avoiding the problems of information loss and gradient disappearance in network transmission, and the output results are more accurate, thereby avoiding the disadvantage of performance degradation in complex background conditions and greatly improving its accuracy.

[0093] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0094] In one embodiment, a device for detecting image forgery based on granularity classification is provided. The device for detecting image forgery based on granularity classification corresponds to the method for detecting image forgery based on granularity classification in the above embodiment. Figure 5 As shown, the image forgery detection device based on granularity classification includes a to-be-detected hierarchical image recognition module 101, an image branch mapping relationship determination module 102, a tampering mask output module 103, a forged image location positioning module 104, and a forgery behavior determination module 105. Detailed descriptions of each functional module are as follows:

[0095] The to-be-detected level image recognition module 101 is configured to determine a detection level of a pre-acquired target image according to preset user requirements, and to identify the to-be-detected level image corresponding to the target image according to the detection level;

[0096] An image branch mapping relationship determination module 102 is configured to determine a detection branch according to the detection level, map the image of the level to be detected with the detection branch, and obtain an image branch mapping relationship;

[0097] a tampering mask output module 103, configured to input the to-be-detected hierarchical image into the detection branch one by one through the image branch mapping relationship, and output the tampering mask of the to-be-detected hierarchical image;

[0098] a forged image location locating module 104 for fusing the tampering mask and the target image to obtain a fused image, and locating the forged image position in the target image according to the fused image;

[0099] The forgery behavior determination module 105 is configured to determine the forgery behavior in the target image according to the forged image position.

[0100] In one embodiment, the to-be-detected level image recognition module 101, when determining the detection level of a pre-acquired target image according to a preset user requirement, is configured to:

[0101] Extracting a demand hierarchy from the user demand;

[0102] Dividing the requirement hierarchy into local image requirements, target image requirements and global image requirements;

[0103] Determining, according to the local image requirement, a detection level of the target image as a local detection level;

[0104] Determining, according to the target image requirement, a detection level of the target image as a target detection level;

[0105] According to the global image requirement, the detection level of the target image is determined to be a global detection level.

[0106] In one embodiment, the to-be-detected level image recognition module 101, when performing the process of identifying the to-be-detected level image corresponding to the target image according to the detection level, is further configured to:

[0107] When the detection level is a local detection level, locally cropping the target image according to the local image requirement to obtain a local image to be detected corresponding to the local detection level;

[0108] When the detection level is a target detection level, extracting a target detection image corresponding to the target image according to the target image requirement, and determining the target detection image as a local target to-be-detected image corresponding to the target detection level;

[0109] When the detection level is a global detection level, the target image is determined as a global image to be detected corresponding to the global detection level.

[0110] In one embodiment, the image branch mapping relationship determining module 102, when mapping the to-be-detected hierarchical image with the detection branch to obtain the image branch mapping relationship, is configured to:

[0111] Dividing the detection branch into a local detection branch, a target detection branch and a global detection branch;

[0112] The hierarchical images in the hierarchical images to be detected are respectively matched one-to-one with the local detection branch, the target detection branch and the global detection branch;

[0113] The relationship between the corresponding hierarchical image and the detection branch is determined as the image branch mapping relationship.

[0114] In one embodiment, the tampering mask output module 103, when inputting the to-be-detected hierarchical image into the detection branch one by one through the image branch mapping relationship and outputting the tampering mask of the to-be-detected hierarchical image, is configured to:

[0115] Extracting a hierarchical detection branch corresponding to the hierarchical image to be detected according to the image branch mapping relationship, and extracting multi-scale image features of the hierarchical image to be detected using a feature extraction model in the hierarchical detection branch;

[0116] An image feature corresponding to the minimum scale in the multi-scale image features is used as a target scale image feature, an upsampling operation is performed on the target scale image feature, and a target mask feature corresponding to the upsampled target scale image feature is output through a preset convolution head;

[0117] The target mask feature is fused with the preset initial mask feature to obtain a mask fusion feature, and the mask fusion feature is supervised by a preset true label to obtain a target-level tampering mask;

[0118] The target-level tampering mask is updated to the initial mask feature, the image feature of the next scale corresponding to the minimum scale is updated to the image feature corresponding to the minimum scale, and the process returns to the step of using the image feature corresponding to the minimum scale in the multi-scale image features as the target-scale image feature, until the image features corresponding to all scales in the multi-scale image features are used as the target-scale image feature, and the tampering mask corresponding to the image of the level to be detected is output.

[0119] In one embodiment, the forged image location locating module 104, when fusing the tampering mask and the target image to obtain a fused image, is configured to:

[0120] determining a hierarchical tampering image of the hierarchical image to be detected according to the tampering mask;

[0121] Converting the target image into a target binary image;

[0122] An intersection operation is performed on the hierarchical tampered image and the target binary image to obtain a fused image.

[0123] In one embodiment, the forgery behavior determination module 105, when determining the forgery behavior in the target image based on the forged image position, is configured to:

[0124] identifying a forged image in the target image according to the forged image position;

[0125] determining a forgery type in the target image based on the forged image;

[0126] The forgery behavior in the target image is determined according to the forgery type.

[0127] The present invention provides an image forgery detection device based on granularity grading. According to preset user needs, the detection strategy can be optimized for specific scenarios and targets, thereby improving the pertinence and accuracy of detection. Image forgery detection is carried out in a hierarchical manner, and forged images are divided into multiple scales according to needs. Images within a specific scale are detected, and it is possible to identify in detail which areas have forged or tampered behaviors. The detected mask contains multi-scale information of the image to be detected, avoiding the problems of information loss and gradient vanishing in network transmission, and the output result is more accurate, thereby avoiding the disadvantage of performance degradation in complex background conditions and greatly improving its accuracy.

[0128] The specific limitations of the granularity-based image forgery detection device can be found in the limitations of the intelligent question-answering method above and will not be repeated here. Each module in the granularity-based image forgery detection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each of these modules.

[0129] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a granularity-based image forgery detection method.

[0130] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the client side of a granularity-based image forgery detection method.

[0131] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0132] Determine the detection level of the pre-acquired target image according to preset user requirements, and identify the image of the to-be-detected level corresponding to the target image according to the detection level;

[0133] Determine a detection branch according to the detection level, and map the image of the level to be detected with the detection branch to obtain an image branch mapping relationship;

[0134] Inputting the to-be-detected hierarchical image into the detection branch one by one through the image branch mapping relationship, and outputting the tampering mask of the to-be-detected hierarchical image;

[0135] fusing the tampering mask and the target image to obtain a fused image, and locating the forged image position in the target image according to the fused image;

[0136] The forgery behavior in the target image is determined according to the forged image position.

[0137] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0138] Determine the detection level of the pre-acquired target image according to preset user requirements, and identify the image of the to-be-detected level corresponding to the target image according to the detection level;

[0139] Determine a detection branch according to the detection level, and map the image of the level to be detected with the detection branch to obtain an image branch mapping relationship;

[0140] Inputting the to-be-detected hierarchical image into the detection branch one by one through the image branch mapping relationship, and outputting the tampering mask of the to-be-detected hierarchical image;

[0141] fusing the tampering mask and the target image to obtain a fused image, and locating the forged image position in the target image according to the fused image;

[0142] The forgery behavior in the target image is determined according to the forged image position.

[0143] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0144] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0145] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0146] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. If any software tools or components other than those of the company appear in the application embodiments, they are merely used for illustration and do not represent actual use. Although the present invention has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above-mentioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for detecting image forgery based on granularity classification, characterized in that: include: Determine the detection level of the pre-acquired target image according to preset user requirements, and identify the level image to be detected corresponding to the target image according to the detection level; Determine a detection branch according to the detection level, map the level image to be detected with the detection branch, and obtain an image branch mapping relationship; Inputting the to-be-detected hierarchical images into the detection branches one by one through the image branch mapping relationship, and outputting the tampering mask of the to-be-detected hierarchical images; fusing the tampering mask and the target image respectively to obtain a fused image, and locating the forged image position in the target image according to the fused image; The forgery behavior in the target image is determined by the forged image position.

2. The image forgery detection method based on granularity classification according to claim 1, characterized in that: The step of determining the detection level of the pre-acquired target image according to a preset user requirement includes: Extracting a demand hierarchy from the user demand; Dividing the demand level into local image demand, target image demand and global image demand; Determining, according to the local image requirement, that the detection level of the target image is a local detection level; Determining, according to the target image requirement, a detection level of the target image as a target detection level; According to the global image requirement, the detection level of the target image is determined to be a global detection level.

3. The image forgery detection method based on granularity classification as claimed in claim 2, characterized in that: The step of identifying the to-be-detected level image corresponding to the target image according to the detection level includes: When the detection level is a local detection level, locally cropping the target image according to the local image requirement to obtain a local image to be detected corresponding to the local detection level; When the detection level is a target detection level, extracting a target detection image corresponding to the target image according to the target image requirement, and determining the target detection image as a local target to-be-detected image corresponding to the target detection level; When the detection level is a global detection level, the target image is determined as a global image to be detected corresponding to the global detection level.

4. The image forgery detection method based on granularity classification according to claim 1, characterized in that: The mapping of the to-be-detected hierarchical image with the detection branch to obtain an image branch mapping relationship includes: Dividing the detection branch into a local detection branch, a target detection branch and a global detection branch; The hierarchical images in the hierarchical images to be detected are respectively matched one by one with the local detection branch, the target detection branch and the global detection branch; The relationship between the corresponding hierarchical image and the detection branch is determined as the image branch mapping relationship.

5. The image forgery detection method based on granularity classification as claimed in claim 1, characterized in that: The step of inputting the to-be-detected hierarchical images into the detection branches one by one through the image branch mapping relationship and outputting the tampering mask of the to-be-detected hierarchical images comprises: Extracting the hierarchical detection branch corresponding to the hierarchical image to be detected according to the image branch mapping relationship, and extracting the multi-scale image features of the hierarchical image to be detected by using the feature extraction model in the hierarchical detection branch; An image feature corresponding to the minimum scale in the multi-scale image features is used as a target scale image feature, an upsampling operation is performed on the target scale image feature, and a target mask feature corresponding to the up-sampled target scale image feature is output through a preset convolution head; The target mask feature is fused with the preset initial mask feature to obtain a mask fusion feature, and the mask fusion feature is supervised by a preset true label to obtain a target level tampering mask; The target level tampering mask is updated to the initial mask feature, the image feature of the next scale corresponding to the minimum scale is updated to the image feature corresponding to the minimum scale, and the process returns to the step of using the image feature corresponding to the minimum scale in the multi-scale image features as the target scale image feature, until all the image features corresponding to the scales in the multi-scale image features are used as the target scale image feature, and the tampering mask corresponding to the image of the level to be detected is output.

6. The image forgery detection method based on granularity classification as claimed in claim 5, characterized in that: The step of fusing the tampering mask and the target image to obtain a fused image includes: Determining a hierarchical tampering image of the hierarchical image to be detected according to the tampering mask; Converting the target image into a target binary image; An intersection operation is performed on the hierarchical tampered image and the target binary image to obtain a fused image.

7. The image forgery detection method based on granularity classification according to claim 1, characterized in that: The determining the forgery behavior in the target image by using the forged image position includes: identifying a forged image in the target image according to the forged image position; determining a forgery type in the target image according to the forged image; The forgery behavior in the target image is determined according to the forgery type.

8. An image forgery detection device based on granularity classification, characterized in that: include: The to-be-detected level image recognition module is used to determine the detection level of the pre-acquired target image according to preset user requirements, and to identify the to-be-detected level image corresponding to the target image according to the detection level; An image branch mapping relationship determination module is used to determine the detection branch according to the detection level, map the level image to be detected with the detection branch, and obtain an image branch mapping relationship; a tampering mask output module, used to input the to-be-detected hierarchical image into the detection branch one by one through the image branch mapping relationship, and output the tampering mask of the to-be-detected hierarchical image; a forged image position locating module, used to fuse the tampering mask and the target image respectively to obtain a fused image, and locate the forged image position in the target image according to the fused image; The forgery behavior determination module is used to determine the forgery behavior in the target image according to the forged image position.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the image forgery detection method based on granularity classification are implemented as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the image forgery detection method based on granularity classification according to any one of claims 1 to 7 are implemented.

Images