Tamper detection method and apparatus

Abstract

Provided in the embodiments of the present description are a tamper detection method and apparatus. The method comprises: acquiring position information of a target region in a card image to be subjected to detection, wherein the target region is a region where a key object in said card image is located; performing feature extraction on said card image by means of one or more first extraction layers, so as to obtain one or more first feature maps; on the basis of the one or more first feature maps and the position information, determining a region feature map corresponding to the target region; by means of a first classification network and on the basis of the region feature map, obtaining a first detection result indicating whether the target region is a tampered region; and at least on the basis of the first detection result, determining whether the card image includes a tamper detection result for the tampered region.

Description

Tamper detection method and device

[0001] The present application claims priority to the Chinese patent application No. 202411329109X, filed on September 23, 2024, and entitled "A tamper detection method and device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present specification relates to the technical field of data processing, and in particular to a tamper detection method and device. BACKGROUND

[0003] With the rapid development of the Internet, people can check relevant cards (including bank cards, various types of identity cards, etc.) through an online checking system provided by a relevant institution online, and only after the checking is passed, can the online business services provided by the relevant institution be enjoyed.

[0004] At present, there are some behaviors of tampering with cards (bank cards, identity cards, etc.) (for example, physically removing the information of the card itself by paper strips and / or other face images, handwriting pollution, etc., and for example, modifying the information of the card itself in the card image by image modification technology) in order to cheat the online checking system of the relevant institution. The above-mentioned behaviors greatly test the security of the online checking system of the relevant institution. Therefore, how to provide a method for detecting the tampered area in the card has become a problem to be solved.

[0005] SUMMARY

[0006] One or more embodiments of the present specification provide a tamper detection method and device to realize accurate detection of the tampered area of the card.

[0007] According to a first aspect, a tamper detection method is provided, comprising:

[0008] obtaining position information of a target area in a card image to be detected, wherein the target area is a region where a key object in the card image is located;

[0009] performing feature extraction on the card image through one or more first extraction layers to obtain one or more first feature maps;

[0010] determining a region feature map corresponding to the target area based on the one or more first feature maps and the position information;

[0011] obtaining a first detection result of whether the target area is a tampered area based on the region feature map through a first classification network;

[0012] At least based on the first detection result, determine a tampering detection result of whether the card image contains a tampered region.

[0013] According to a second aspect, a tampering detection apparatus is provided, comprising:

[0014] A first obtaining module configured to obtain position information of a target region in a card image to be detected, wherein the target region is a region where a key object in the card image is located;

[0015] A first obtaining module configured to obtain position information of a target region in a card image to be detected, wherein the target region is a region where a key object in the card image is located;

[0016] A first determining module configured to determine a region feature map corresponding to the target region based on the one or more first feature maps and the position information;

[0017] A second obtaining module configured to obtain, by a first classification network, a first detection result of whether the target region is a tampered region based on the region feature map;

[0018] A second determining module configured to determine a tampering detection result of whether the card image contains a tampered region at least based on the first detection result.

[0019] According to a third aspect, a computer readable storage medium is provided, which stores a computer program, and when the computer program is executed in a computer, the computer executes the method of the first aspect.

[0020] According to a fourth aspect, a computing device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method of the first aspect is implemented.

[0021] According to the tamper detection method and device provided in the embodiments of the present specification, position information of a region where a target area key object in a card image to be detected is obtained; feature extraction is performed on the card image through one or more first extraction layers to obtain one or more first feature maps; based on the one or more first feature maps and the position information, a region feature map corresponding to the target area is determined; through a first classification network, based on the region feature map, a first detection result of whether the target area is a tampered region is obtained; and based on at least the first detection result, a tamper detection result of whether the card image contains a tampered region is determined. In the above process, for the target area (i.e., the region where the key object with a higher possibility of being tampered) in the card image, based on the corresponding region feature map, a first detection result of whether the target area is a tampered region is determined, which can realize the key attention and detection of the local key region in the card image, and then based on the first detection result of the local key region in the image, a tamper detection result of whether the card image globally contains a tampered region is determined, which realizes the determination of the tamper detection result of the card image globally, and improves the accuracy of the tamper detection result to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creating laborious work.

[0023] FIG. 1A is an exemplary card image containing a tampered region provided by the present specification;

[0024] FIG. 1B is an exemplary card image containing a tampered region provided by the present specification;

[0025] FIG. 1C is an exemplary card image containing a tampered region provided by the present specification;

[0026] FIG. 2 is a schematic diagram of the implementation framework of one embodiment disclosed by the present specification;

[0027] FIG. 3 is a schematic diagram of the flow of the tamper detection method provided by the embodiment;

[0028] FIG. 4 is another schematic diagram of the flow of the tamper detection method provided by the embodiment;

[0029] FIG. 5 is a schematic diagram of the structure of the tamper detection model provided by the embodiment;

[0030] FIG. 6 is a schematic diagram of the structure of the tamper detection model provided by the embodiment;

[0031] FIG. 7 is a schematic diagram of the structure of the first edge enhancement subnetwork provided by the embodiment;

[0032] FIG. 8 is another structural schematic diagram of the tampering detection model provided by the embodiments;

[0033] FIG. 9 is another structural schematic diagram of the tampering detection model provided by the embodiments;

[0034] FIG. 10 is a schematic diagram of a tampering detection process provided by the embodiments;

[0035] FIG. 11 is a schematic block diagram of a tampering detection device provided by the embodiments. DETAILED DESCRIPTION

[0036] The technical solutions of the embodiments of the present specification will be described in detail below with reference to the accompanying drawings.

[0037] At present, there are some behaviors of tampering with cards (bank cards, identity cards, etc.) (for example, physically unmasking the information of the card itself in the card image through paper strips and / or other face images, handwriting pollution, etc., and for example, modifying the information of the card itself in the card image through image modification technology) in order to deceive the online card verification system of some institutions. Among them, FIG. 1A shows a schematic diagram of the result corresponding to the tampering method of physically unmasking the information of the card itself through a paper strip, wherein the area pointed by the arrow is the tampered area in the card; FIG. 1B shows a schematic diagram of the result corresponding to the tampering method of unmasking the information of the card itself through note pollution, wherein the area pointed by the arrow is the tampered area in the card; FIG. 1C shows a schematic diagram of the result corresponding to the tampering method of unmasking the information of the card itself through image modification technology, wherein the information in the card pointed by the arrow on the left is the real information, and the information in the card pointed by the arrow on the right is the tampered information, i.e., the tampered area in the card.

[0038] In order to ensure the safety of the aforementioned online card verification system of the institution and improve the defense capability of the aforementioned institution against attacks by tampered card images, the present specification discloses a tampering detection method and device to accurately detect the tampered area in the card, thereby improving the defense capability of the aforementioned institution against attacks by tampered card images.

[0039] Among them, FIG. 2 shows a schematic diagram of an implementation scenario according to an embodiment disclosed by the present specification. In this implementation scenario, a card image to be detected is obtained, which can be an image of any type of card, such as but not limited to a bank card, a credit card, various types of identity cards, such as an identity card, a driver's license, and a passport, etc.

[0040] Then, position information of a target region in the card image is obtained, where the target region can be a region where a key object in the card is located, and the key object is an object in the card that is more likely to be tampered with, such as a specified string and / or a face image, etc., and the specified string can include an identity (such as an ID), a name, a bank card number, etc. Then, the card image is input into a network for feature extraction (hereinafter referred to as a main branch network), and one or more first feature maps are obtained by performing feature extraction on the card image through one or more first extraction layers in the network. Then, based on the one or more first feature maps and the position information of the target region, a region feature map corresponding to the target region is determined. Then, a first detection result of whether the target region is a tampered region is obtained based on the region feature map through a first classification network. Then, based on the first detection result, a tampering detection result of whether the card image contains a tampered region is determined.

[0041] In the above process, the first detection result of whether the target region (i.e., the region where the key object with a higher possibility of being tampered with is located) in the card image is a tampered region is determined based on the region feature map corresponding to the target region, which can realize the key attention and detection of the local key region in the card image. Then, based on the first detection result of the local key region in the image, the tampering detection result of whether the card image contains a tampered region is determined, which can realize the determination of the global tampering detection result of the card image and improve the accuracy of the tampering detection result to a certain extent.

[0042] The tampering detection method provided in the specification will be described in detail below in conjunction with specific embodiments.

[0043] FIG. 3 shows a flowchart of a tampering detection method in one embodiment of the specification. The method is executed by an electronic device, which can be implemented by any device, apparatus, platform, device cluster, etc. with computing and processing capabilities. In the tampering detection process, as shown in FIG. 3, the method includes the following steps S310-S350:

[0044] In step S310, position information of a target region in the card image to be detected is acquired. The target region can be a region where a key object is located in the card image. In some possible examples, the region where the key object is located includes a region where a specified string is located and / or a region where a face image is located. Correspondingly, the key object can include a specified string and / or a face image, and the like, which are generally more likely to be tampered with in a card. For example, when the card image is an image corresponding to a bank card or a credit card, the key object can include, but is not limited to, at least one of a bank card number and a bank name of a bank to which the bank card belongs, and the like. When the card image is an image corresponding to an identity card, the key object can include, but is not limited to, at least one of an identity (for example, an ID), a name, an address, and the like, and a face image, and the like.

[0045] In some possible examples, the position information of the target region can be position information of the target region detected by another device from a card image, which is acquired by the electronic device from the another device.

[0046] In some possible examples, the electronic device can detect the target region in the card image based on a preset detection algorithm. In step S310, the method can include the following steps 11-14.

[0047] In step 11, the card image is detected based on a preset detection algorithm to obtain position information of each candidate target region in the card image. The candidate target region includes a region suspected to be a region where a key object is located, which is detected by the preset detection algorithm from the card image. For example, the region suspected to be the region where the key object is located can include a region where a string and / or a face image is located in the card image.

[0048] In some possible examples, the preset detection algorithm can be any algorithm that can detect a string and / or a face region in the related art. For example, the algorithm can be a neural network model trained based on sample images and corresponding label data (including position information of a region where a string and / or a face is located in the sample image) in advance. For example, the sample images can include images obtained by photographing any card, or any image including a string and / or a face.

[0049] Exemplarily, the preset detection algorithm can be, but is not limited to, a model based on a DB (Differentiable Binarization) algorithm to achieve accurate detection of a character string in an image, for example, a DBNet++ model; can also be a special face detection model to achieve accurate detection of a face in an image; and can also be a detection algorithm based on a Faster R-CNN (Faster Region-based Convolutional Network).

[0050] The electronic device can detect the card image based on the preset detection algorithm to obtain position information of each candidate target region in the card image, i.e., a region suspected to be a key object. In one case, the position information of each candidate target region can be displayed in the card image in the form of a labeled box.

[0051] Then, in step 12, a specified region alignment algorithm is used to obtain a feature map corresponding to each candidate target region mapped to a first size based on the position information of each candidate target region. The first size is suitable for the second classification network and can be equal to the size of the input data of the second classification network.

[0052] In some possible examples, the electronic device can input the position information of each candidate target region and the card image into the specified region alignment algorithm to process each candidate target region in the card image based on the position information of each candidate target region by using the specified region alignment algorithm, so as to obtain a feature map corresponding to each candidate target region mapped to a first size.

[0053] In yet some possible examples, the electronic device can input the position information of each candidate target region and a specified feature map corresponding to the card image into the specified region alignment algorithm to process a feature corresponding to each candidate target region in the specified feature map corresponding to the card image based on the position information of each candidate target region by using the specified region alignment algorithm, so as to obtain a feature map corresponding to each candidate target region mapped to a first size. Exemplarily, in a case where the preset detection algorithm is a neural network model used for target detection, the specified feature map corresponding to the card image can be a feature map of a specified extraction layer in the neural network model, or a feature map fused after features of at least two specified extraction layers in the neural network model.

[0054] The specified region alignment algorithm includes, for example, an ROI (Region of Interest) Align algorithm.

[0055] At step 13, a classification result of whether each candidate target region is a target region is determined based on the first size feature map corresponding to each candidate target region by a second classification network. In this step, the electronic device can input the first size feature map corresponding to each candidate target region into the second classification network respectively, and process the first size feature map corresponding to each candidate target region by the second classification network to determine the classification result of whether each candidate target region is a target region. The second classification network can be any type of pre-trained network for classification (distinguishing whether the indicated region in the corresponding feature map is a target region, i.e., the region where the key object is located) such as a multi-layer perception or a classifier based on other classification algorithms.

[0056] Then, at step 14, the position information of the target region is determined from the position information of each candidate target region based on the aforementioned classification result. In this step, the classification result can indicate which candidate target region is the region where the key object is located, i.e., the target region. Accordingly, the electronic device determines the position information of the target region, i.e., the position information of the region where the key object is located, from the position information of each candidate target region based on the aforementioned classification result. In some examples, the target region can be one or more.

[0057] At step S320, feature extraction is performed on the card image by one or more first extraction layers to obtain one or more first feature maps. The one or more first extraction layers can belong to a network for feature extraction, such as the main branch network in the tamper detection model mentioned later, and the one or more first extraction layers are arranged in series to perform feature extraction on the input image. In this step, the electronic device inputs the card image into the one or more first extraction layers to perform feature extraction on the card image by the one or more first extraction layers to obtain one or more first feature maps. When there are multiple first feature maps, they can be multi-scale feature maps, and there is a corresponding relationship between the first feature map and the first extraction layer.

[0058] After obtaining the one or more first feature maps, at step S330, a region feature map corresponding to the target region is determined based on the one or more first feature maps and the aforementioned position information.

[0059] In some possible implementations, the electronic device can determine the region feature map corresponding to the target region based on a specified first feature map in the one or more first feature maps and the aforementioned position information of the target region. Specifically, the electronic device can convert the position information of the target region from the size space of the card image to the size space of the specified feature map based on the size ratio between the specified first feature map and the card image, and then determine the region feature map corresponding to the target region from the specified feature map based on the position information in the size space of the specified feature map.

[0060] In yet some possible implementation, in order to obtain a detection result with better accuracy, a region feature map corresponding to the target region can be determined based on the first feature maps of multiple scales, so that the obtained region feature map corresponding to the target region includes both global information and local detailed information, and the accuracy of the detection result is improved. Specifically, the first feature maps mentioned above are multiple; correspondingly, step S330 can include 21-22:

[0061] In step 21, at least two specified first feature maps in the multiple first feature maps are fused to obtain a fused feature map. In this step, at least two specified first feature maps are determined from the multiple first feature maps, and then the at least two specified first feature maps are size-unified, for example, the smaller size first feature maps in the at least two specified first feature maps are up-sampled according to the size of the largest size first feature map in the at least two specified first feature maps, to obtain the size-unified at least two specified first feature maps; or for example, the smaller size first feature maps in the at least two specified first feature maps are down-sampled according to the size of the smallest size first feature map in the at least two specified first feature maps, to obtain the size-unified at least two specified first feature maps.

[0062] After obtaining the size-unified at least two specified first feature maps, the size-unified at least two specified first feature maps are fused (for example, pixel-by-pixel summation or splicing, etc.) to obtain a fused feature map.

[0063] Then, in step 22, based on the position information of the target region and the size ratio between the fused feature map and the card image, a region feature map corresponding to the target region is determined from the fused feature map.

[0064] In this step, the electronic device converts the position information of the target region from the size space of the card image to the size space of the fused feature map based on the size ratio between the fused feature map and the card image, and then determines a region feature map corresponding to the target region from the fused feature map based on the position information converted to the size space of the fused feature map. The information contained in such a region feature map is more abundant, which is more beneficial for subsequent tampering region detection.

[0065] After obtaining the region feature map of the target region, in step S340, a first detection result of whether the target region is a tampered region is obtained based on the region feature map by a first classification network. The first classification network can be any type of classification network. For example, the first classification network can be a multi-layer perception machine used to determine whether the input feature map is a tampered region. In some possible examples, the electronic device can process the region feature map by the first classification network to obtain the first detection result of whether the target region is a tampered region. In some examples, the first detection result can be a score value indicating whether the target region is a tampered region. The larger the score value, the greater the possibility that the target region is a tampered region.

[0066] In yet some possible examples, in order to improve the accuracy of the subsequent detection result, in step S340, steps 31-32 can be included.

[0067] In step 31, the region feature map is processed by using a specified region alignment algorithm to obtain a feature map corresponding to the region feature map and mapped to a second size. The second size conforms to the size of the input data of the first classification network. The specified region alignment algorithm is, for example, a ROI Align algorithm.

[0068] Then, in step 32, the first detection result of whether the target region is a tampered region is obtained based on the second size feature map corresponding to the region feature map by the first classification network. In this step, the electronic device inputs the second size feature map corresponding to the region feature map into the first classification network, and the first classification network processes the second size feature map corresponding to the region feature map to obtain the first detection result of whether the target region is a tampered region.

[0069] In some possible examples, in the case where the target region is multiple, different target regions correspond to different first detection results.

[0070] Then, in step S350, a tampering detection result of whether the card image contains a tampered region is determined based on at least the first detection result.

[0071] In some possible examples, in the case where any first detection result indicates that the target region corresponding thereto is a tampered region, it can be determined that the tampering detection result is that the card image contains a tampered region.

[0072] In this embodiment, for the target region (i.e., the region where the key object with a high possibility of being tampered with is located) in the card image, based on the corresponding region feature map, whether it is the first detection result of the tampered region is determined, which can realize the key attention and detection of the local key region in the card image, and then based on the first detection result of the local key region in the image, the tampering detection result of whether the card image globally contains a tampered region is determined, the determination of the global tampering detection result of the card image is realized, and the accuracy of the tampering detection result is improved to a certain extent.

[0073] In the above process, based on the region feature map corresponding to each target region in the card image, i.e., the region where the key object is located, the feature map obtained after the region alignment algorithm is used for tampering detection, and for some small tampered regions in the card image (for example, regions tampered by small handwriting and regions tampered by small foreign objects, etc.), the accuracy of the detection result is better improved.

[0074] FIG. 4 shows a flowchart of a tampering detection method in another embodiment of the present specification. The method is executed by an electronic device, which can be realized by any device, equipment, platform, device cluster, etc. with computing and processing capabilities.

[0075] In the tampering detection process shown in FIG. 4, the local region detection result that needs to be focused on in the card image and the global segmentation detection result of the card image can be combined to determine whether the card image contains a tampered region, so as to better improve the accuracy of the tampering detection result, realize the global attention and local detail region attention of the card image. In the tampering detection process shown in FIG. 4, the tampering detection model needs to be used to realize the overall detection of the card image.

[0076] The training process of the tampering detection model will be introduced first. The tampering detection model includes a main branch network, a reversible branch network, a first edge detection network, and a first region detection network. The main branch network includes a plurality of first extraction layers (i.e., the first extraction layer mentioned above) connected in series, and the reversible branch network includes a plurality of second extraction layers connected in series. The first edge detection network pays more attention to the edge information of the tampered region in the image, and is a network for detecting the edge information of the tampered region in the image; the first region detection network pays more attention to the overall region structure information of the tampered region in the image, and is a network for detecting the location of the tampered region in the image.

[0077] Before training the tampering detection model, a training dataset S for training the tampering detection model can be constructed, the training dataset S includes a plurality of sample images X and label data Y thereof. The sample images X can be card image containing tampering regions (i.e., the sample image contains tampering regions) or card image not containing tampering regions (i.e., the sample image does not contain tampering regions). The tampering region can include a region in the card that is occluded by a paper strip (as shown in FIG. 1A) and / or other face images, handwriting pollution (as shown in FIG. 1B), and the like, which are physically occluded by tampering; or can include a region in the card that is modified by image modification technology (as shown in FIG. 1C).

[0078] In some examples, the label data corresponding to each sample image can include a label segmentation map generated based on the tampering region in the sample image; and can also include a label edge map generated based on the tampering region in the sample image.

[0079] The label segmentation map can be a binary mask image. For a sample image containing a tampering region, the binary mask image can indicate the location of the tampering region in the corresponding sample image. For example, the pixel value of a pixel point indicating the tampering region can be a first value (e.g., 1), and the pixel value of a pixel point indicating a non-tampering region can be a second value (e.g., 0). For a sample image not containing a tampering region, the binary mask image can indicate that the corresponding sample image does not contain a tampering region, and the pixel value of each pixel point can be the second value.

[0080] The label edge map can also be a binary image. For a sample image containing a tampering region, the binary image can indicate the edge information of the tampering region in the corresponding sample image. For example, the pixel value of a pixel point indicating the edge of the tampering region can be a first value, and the pixel value of a pixel point indicating the edge of a non-tampering region can be a second value. For a sample image not containing a tampering region, the binary image can indicate that the corresponding sample image does not contain a tampering region, and the pixel value of each pixel point can be the second value.

[0081] In yet other examples, the label data corresponding to each sample image can also include a label value indicating whether the sample image contains a tampering region. For example, a label value of a first value (e.g., 1) indicates that the sample image contains a tampering region, and a label value of a second value (e.g., 0) indicates that the sample image does not contain a tampering region.

[0082] After obtaining the training data set S, the tampering detection model can be trained based on each sample image and its label data in the training data set S. It can be understood that the process of training the tampering detection model based on each sample image and its label data is similar, and the process of training the tampering detection model will be introduced below by taking any first sample image Xj and its first label data Yj in the training data set S as an example. The process of training the tampering detection model using other sample images and their label data can be referred to the process of training the tampering detection model using the first sample image and its first label data. Specifically, the process of training the tampering detection model can include the following steps 41-46:

[0083] In step 41, the first sample image and its first label data are obtained, wherein the first label data is used to indicate whether the first sample image contains a tampered region.

[0084] In some possible examples, the first label data can include a label segmentation map and a label edge map generated based on the first sample image. The label segmentation map can indicate whether the first sample image (i.e., a card certificate) contains a tampered region, and in the case that the first sample image contains a tampered region, the label segmentation map in its first label data can indicate the location of the tampered region contained in the first sample image. The label edge map can indicate whether the first sample image contains a tampered region, and in the case that the first sample image contains a tampered region, the label edge map in its first label data can indicate the edge position of the tampered region contained in the first sample image. Wherein, the card certificate can include but not limited to: bank cards, various types of identity cards, etc.

[0085] After the electronic device obtains the first sample image and its first label data, in step 42, the first sample image is feature-extracted based on a plurality of first extraction layers in series in the main branch network, to obtain a plurality of second feature maps corresponding to the plurality of specified first extraction layers.

[0086] In this step, the electronic device can input the first sample image into the main branch network, and feature-extract the first sample image based on a plurality of first extraction layers in series in the main branch network, to obtain a plurality of second feature maps corresponding to the plurality of specified first extraction layers. Wherein, in the plurality of first extraction layers in series, the input of the first first extraction layer is the first sample image, and the input of each non-first first extraction layer is the output of the previous first extraction layer.

[0087] In some possible implementations, each first extraction layer of the main branch network can include, but is not limited to, a convolution operation, a pooling operation, a batch normalization operation, and other operations for feature extraction of an image. In some examples, the main branch network can be a Res2Net-50 structure-based extraction network to implement feature extraction of the first sample image, to obtain a plurality of second feature maps corresponding to a plurality of specified first extraction layers, and the second feature maps corresponding to different specified first extraction layers can have different sizes.

[0088] The aforementioned specified first extraction layers can at least include each first extraction layer corresponding to each third extraction layer in the set reversible branch network mentioned later and each first extraction layer at the same layer as each third extraction layer. For example, as shown in FIG. 5, the main branch network can include five first extraction layers, including extraction layer 11, extraction layer 12, extraction layer 13, extraction layer 14, and extraction layer 15 as shown in FIG. 5. The specified first extraction layers can include each non-first first extraction layer, for example, including extraction layer 12, extraction layer 13, extraction layer 14, and extraction layer 15 as shown in FIG. 5.

[0089] In step 43, the first sample image is processed based on the reversible branch network to obtain a plurality of third feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes, through a third extraction layer in the plurality of second extraction layers, performing feature extraction on a third feature map corresponding to a previous layer and a set second feature map corresponding to the third extraction layer to obtain an output feature map, and based on the output feature map and the second feature map of the first extraction layer at the same layer as the third extraction layer, obtaining a corresponding third feature map, the third extraction layer being a specified layer in the plurality of second extraction layers starting from the second second extraction layer.

[0090] In this step, the electronic device inputs the first sample image into the reversible branch network, and processes the first sample image based on a plurality of serially connected second extraction layers included in the reversible branch network, wherein the plurality of serially connected second extraction layers include one or more third extraction layers, the input of each third extraction layer including a third feature map corresponding to a previous extraction layer and a set second feature map corresponding to the third extraction layer; the input of each second extraction layer other than the third extraction layer in the plurality of serially connected second extraction layers can be the first sample image or a third feature map corresponding to a previous extraction layer.

[0091] In some possible examples, for each third extraction layer in the plurality of second extraction layers connected in series (taking the third extraction layer P as an example), the third extraction layer P includes a fusion unit P1 and an extraction unit P2, and the process of performing feature extraction on the third feature map a1 corresponding to the previous layer of the third extraction layer and the second feature map b1 set to correspond to the third extraction layer by the third extraction layer to obtain the output feature map of the third extraction layer can include: adjusting the sizes (for example, up-sampling and / or down-sampling) of the third feature map a1 corresponding to the previous layer of the third extraction layer and the second feature map b1 set to correspond to the third extraction layer, respectively, to unify the sizes of the third feature map a1 corresponding to the previous layer of the third extraction layer and the second feature map b1 set to correspond to the third extraction layer.

[0092] Then, the third feature map a2 after the size adjustment of the previous layer of the third extraction layer is subjected to first preset processing by the fusion unit P1 to obtain a corresponding processed third feature map a3, and the second feature map b2 set to correspond to the third extraction layer after the size adjustment is subjected to second preset processing to obtain a corresponding processed second feature map b3; the processed third feature map a3 and the processed second feature map b3 are subjected to first fusion processing to obtain a fusion feature map c of the third extraction layer; and the fusion feature map c is subjected to feature extraction by the extraction unit P2 to obtain the output feature map of the third extraction layer P.

[0093] For example, the first preset processing can include but is not limited to convolution operation and batch normalization operation, and the second preset processing can include but is not limited to linear processing, batch normalization operation, and difference processing, etc.; and the first fusion processing can include pixel-by-pixel addition of the processed third feature map and the processed second feature map, that is, adding the value of the pixel at the a-th row and the b-th column in the processed third feature map to the value of the pixel at the a-th row and the b-th column in the processed second feature map.

[0094] The extraction unit P2 can be any extraction unit that can perform feature extraction in the related art, for example, a feature extraction unit based on a residual network, for example, a feature extraction unit based on a ConvNeXt network, etc.

[0095] Then, after obtaining the output feature map of the third extraction layer P, the second feature map of the first extraction layer at the same layer as the third extraction layer P is subjected to preset reversible processing to obtain a reversibly processed feature map, and the output feature map of the third extraction layer P and the reversibly processed feature map are subjected to second fusion processing to obtain the third feature map corresponding to the third extraction layer P.

[0096] The preset reversible processing can refer to an operation in which the corresponding output after the inverse operation of the preset reversible processing can obtain the corresponding input. The reversible processing can include but is not limited to multiplying the second feature map of the first extraction layer at the same layer as the third extraction layer P with a preset coefficient, adding the second feature map of the first extraction layer at the same layer as the third extraction layer P with specified data, and the like. The second fusion processing can be pixel-by-pixel addition of the output feature map of the third extraction layer P and the feature map after the reversible processing.

[0097] In some examples, the structure of the other second extraction layer can be similar to the structure of the third extraction layer, and the set second feature map corresponding to the other second extraction layer can be empty. In some other examples, the structure of the other second extraction layer can be similar to the structure of the first extraction layer.

[0098] In some possible examples, the number of the plurality of second extraction layers of the reversible branch network can be the same as or different from the number of the plurality of first extraction layers of the main branch network. In some other examples, the number of the plurality of second extraction layers of the reversible branch network can be less than the number of the plurality of first extraction layers of the main branch network. As shown in FIG. 5, the number of the plurality of second extraction layers of the reversible branch network can be the same as the number of the plurality of first extraction layers of the main branch network, both of which are 5 layers. As shown in FIG. 5, the reversible branch network includes extraction layer 21, extraction layer 22, extraction layer 23, extraction layer 24, and extraction layer 25.

[0099] In some implementations, the third extraction layer can be set according to requirements. For example, the third extraction layer can include one or more third extraction layers. The third extraction layer can be a higher layer extraction layer in the reversible branch network, or can be a lower layer extraction layer in the reversible branch network. As shown in FIG. 5, the third extraction layer can be the third layer extraction layer to the fifth layer extraction layer in the reversible branch network, i.e., extraction layer 23, extraction layer 24, and extraction layer 25.

[0100] In some possible examples, the third extraction layer is the i-th layer extraction layer in the reversible branch network. The second feature map corresponding to the third extraction layer can be the second feature map of the i+n-th layer first extraction layer in the main branch network, and n is a positive integer. The specific value of n can be set according to requirements and the number of the first extraction layer in the main branch network.

[0101] For example, n can be set to 1 or 2, etc. As shown in FIG. 5, assuming that n is set to 1, for the third extraction layer in the reversible branch network, the second feature map corresponding to the third extraction layer is set to be the second feature map of the 3+n (n is 1) layer first extraction layer in the main branch network, i.e., the second feature map of the fourth layer first extraction layer in the main branch network, i.e., the extraction layer 14 shown in FIG. 5; for the fourth extraction layer in the reversible branch network, the second feature map corresponding to the third extraction layer is set to be the second feature map of the fifth layer first extraction layer in the main branch network, i.e., the extraction layer 15 shown in FIG. 5; and for the fifth extraction layer in the reversible branch network, the second feature map corresponding to the third extraction layer is set to be empty.

[0102] The plurality of specified second extraction layers can be set according to requirements. For example, the plurality of specified second extraction layers include non-first layer second extraction layers in the plurality of second extraction layers. For example, the extraction layers 22, 23, 24, and 25 shown in FIG. 5.

[0103] After obtaining the plurality of third feature maps corresponding to the plurality of specified second extraction layers, the electronic device obtains, in step 44, a second tampered edge map based on the plurality of third feature maps by using the first edge detection network.

[0104] In some possible examples, considering that the third feature map corresponding to a specified second extraction layer contains more local edge detail information in the image as the specified second extraction layer is located at a shallower layer in the reversible branch network, the specified second extraction layer can provide more tampered region edge information; the third feature map corresponding to the specified second extraction layer contains more global information (position-related information of each tampered region) in the image as the specified second extraction layer is located at a deeper layer in the reversible branch network, i.e., the specified second extraction layer can better supervise the positioning of the edges of the tampered region. To enable the first edge detection network to predict more accurate edge information of the tampered region, the electronic device can input the plurality of third feature maps into the first edge detection network after obtaining the plurality of third feature maps corresponding to the plurality of specified second extraction layers, and obtain, by using the first edge detection network, a second tampered edge map based on the plurality of third feature maps, to better mine the edge information of the tampered region.

[0105] In some possible examples, on the basis of the above case, considering the model training efficiency, the plurality of third feature maps can include a feature map corresponding to a target second extraction layer and a feature map corresponding to a last layer second extraction layer of the reversible branch network, the target second extraction layer being located at a layer not higher than a preset layer in the reversible branch network; and correspondingly, in step 44, the method further includes the following step: inputting the feature map corresponding to the target second extraction layer and the feature map corresponding to the last layer second extraction layer into the first edge detection network to determine the second tampered edge map.

[0106] The target second extraction layer in the reversible branch network has a number of layers not higher than a preset number of layers, the corresponding feature map contains more local edge detail information in the image, the last layer of the second extraction layer corresponds to a feature map containing more global information in the image, and the electronic device inputs the feature map corresponding to the target second extraction layer and the feature map corresponding to the last layer of the second extraction layer into the first edge detection network, so that the first edge detection network can mine more accurate edge semantics related to the tampered region from the feature map corresponding to the target second extraction layer containing more local edge detail information in the image and the feature map corresponding to the last layer of the second extraction layer containing more global information under the supervision of the tampered region boundary, and then obtain a second tampered edge map with higher accuracy.

[0107] In an implementation manner, the aforementioned target second extraction layer can be a second layer extraction layer in the reversible branch network. As shown in FIGS. 5 and 6, the feature map corresponding to the target second extraction layer is feature map f22, and the feature map corresponding to the last layer of the second extraction layer is feature map f25.

[0108] Exemplarily, the first edge detection network can be any network model capable of edge detection in the related art. In some possible examples, the structure of the first edge detection network can be as shown in FIG. 6, which can include a first convolutional layer and a second convolutional layer arranged in parallel, wherein the input of the first convolutional layer is the aforementioned feature map corresponding to the target second extraction layer (for example, feature map f12), and the input of the second convolutional layer is the aforementioned feature map corresponding to the last layer of the second extraction layer (for example, feature map f15). The second convolutional layer is followed by an up-sampling layer “U”, which is connected with a concatenation layer “C”; the first convolutional layer is connected with the concatenation layer; and the concatenation layer is sequentially connected with a plurality of third convolutional layers and an activation layer. The size of the convolution kernel of the first convolutional layer and the second convolutional layer can be 1*1.

[0109] Exemplarily, the concatenation layer can be connected with one or more third convolutional layers. In some possible examples, as shown in FIG. 6, the plurality of third convolutional layers can include a plurality of convolutional layers connected in series, specifically including two third convolutional layers 1 with a convolution kernel size of 3*3 and a third convolutional layer 2 with a convolution kernel size of 1*1 connected in series. The aforementioned concatenation layer is provided with a concatenation function (for example, a concat function) to concatenate the two inputs. The activation layer is provided with an activation function (for example, a Sigmoid function). Based on the local edge detail information contained in the feature map corresponding to the target second extraction layer and the global information contained in the feature map corresponding to the last layer of the second extraction layer, the above structure of the first edge detection network can determine a second tampered edge map with higher accuracy.

[0110] Then, in step 45, a plurality of second segmentation maps are obtained by the first region detection network based on the plurality of third feature maps and the second tampered edge map.

[0111] In this step, the electronic device can integrate the second tampered edge map with each second feature map, i.e., each hierarchical feature, through the first region detection network, guide each hierarchical feature to learn the edge information therein through the second tampered edge map, so as to enhance the edge information in each hierarchical feature; and then obtain a plurality of second segmentation images based on each hierarchical feature with enhanced edge information through the first region detection network.

[0112] In some possible examples, as shown in FIG. 5, the first region detection network can include a plurality of first detection sub-networks connected in series (including “CAM1”, “CAM2” and “CAM3” as shown in FIG. 5), and a plurality of first edge enhancement sub-networks corresponding to each specified second extraction layer (including “EFM1” corresponding to extraction layer 22, inputting the second feature map f22 corresponding to extraction layer 22; “EFM2” corresponding to extraction layer 23, inputting the second feature map f23 corresponding to extraction layer 23; “EFM3” corresponding to extraction layer 24, inputting the second feature map f24 corresponding to extraction layer 24; “EFM4” corresponding to extraction layer 25, inputting the second feature map f25 corresponding to extraction layer 25, as shown in FIG. 5); accordingly, in step 45, steps 451-452 can be included as follows:

[0113] In step 451, each first edge enhancement sub-network processes the third feature map and the second tampered edge map corresponding thereto to obtain an edge enhanced feature map of the first edge enhancement sub-network. It can be understood that there is a corresponding relationship between the specified second extraction layer and the first edge enhancement sub-network, and there is a corresponding relationship between the specified second extraction layer and the third feature map, and accordingly, there is a corresponding relationship between the first edge enhancement sub-network and the third feature map.

[0114] In this step, the electronic device combines the third feature map corresponding to each specified second extraction layer with the second tampered edge map to obtain a plurality of image groups, each image group including a third feature map corresponding to a specified second extraction layer and a second tampered edge map; and inputs each image group into the first edge enhancement sub-network corresponding to the specified second extraction layer, so that each first edge enhancement sub-network processes the third feature map and the second tampered edge map corresponding thereto, integrates the edge information of the second tampered edge map with the features in the corresponding third feature map, i.e., injects the edge information of the second tampered edge map into the representation learning of the corresponding third feature map, to enhance the edge feature representation in the corresponding third feature map, and obtains an edge enhanced feature map of each first edge enhancement sub-network.

[0115] In some possible examples, the first edge enhancement sub-network can be an arbitrary neural network that can realize feature fusion integration. For example, in order to realize good integration and obtain edge enhancement feature maps of the first edge enhancement sub-networks with better edge enhancement effects, the first edge enhancement sub-network can introduce a local attention mechanism to realize feature interaction and integration between the corresponding third feature map and the second tampered edge map. As shown in FIG. 7, an example structure diagram of an arbitrary first edge enhancement sub-network k (k is a positive integer) is shown, which includes a down-sampling layer, a first fusion layer, a second fusion layer, a fifth convolution layer, a processing layer based on a local attention mechanism, a third fusion layer, and a sixth convolution layer connected in sequence.

[0116] As shown in FIG. 7, the input of the down-sampling layer is the second tampered edge map; the first fusion layer is configured to perform pixel-by-pixel multiplication processing, and the input of the first fusion layer is the corresponding third feature map and the second tampered edge map after down-sampling by the down-sampling layer; the second fusion layer is configured to perform pixel-by-pixel addition processing, and the input of the second fusion layer is the corresponding third feature map and the output of the first fusion layer; the input of the fifth convolution layer is the output of the second fusion layer, and the size of the convolution kernel of the fifth convolution layer is set to 3*3, for example; the processing layer based on the local attention mechanism can include a pooling sub-layer (for example, a GAP (Global Average Pooling) layer), a convolution sub-layer, and an activation sub-layer (wherein a Sigmoid activation function is configured, for example); the third fusion layer is configured to perform pixel-by-pixel multiplication, and the input of the third fusion layer is the output of the fifth convolution layer and the output of the processing layer based on the local attention mechanism; and the input of the sixth convolution layer is the output of the third fusion layer.

[0117] The electronic device obtains the edge enhancement feature maps of the first edge enhancement sub-networks, and then, in step 452, processes the edge enhancement feature map of the corresponding first edge enhancement sub-network and the corresponding first input data by using an arbitrary target detection sub-network in the plurality of first detection sub-networks to obtain a second segmentation map corresponding to the target detection sub-network, wherein when the target detection sub-network is the first first detection sub-network, the corresponding first input data is the edge enhancement feature map of the specified first edge enhancement sub-network in the plurality of first edge enhancement sub-networks; and when the target detection sub-network is not the first first detection sub-network, the corresponding first input data is the output of the previous first detection sub-network.

[0118] In this step, the electronic device can sequentially take each first detection sub-network as a target detection sub-network according to the serial connection relationship of the plurality of first detection sub-networks, input the edge enhancement feature map of the first edge enhancement sub-network corresponding to the target detection sub-network and the first input data into the target detection sub-network, and process the edge enhancement feature map of the first edge enhancement sub-network corresponding to the target detection sub-network and the corresponding first input data through the target detection sub-network to obtain a second segmentation map corresponding to the target detection sub-network.

[0119] In the case where the target detection sub-network is the first first detection sub-network, the corresponding first input data is the edge enhancement feature map of the specified first edge enhancement sub-network in the plurality of first edge enhancement sub-networks. For example, in the case where the target detection sub-network is the first first detection sub-network, the specified first edge enhancement sub-network can be the first edge enhancement sub-network corresponding to the second feature map corresponding to the last second extraction layer. In the case where the target detection sub-network is not the first first detection sub-network, the corresponding first input data is the output of the previous first detection sub-network.

[0120] By setting a plurality of serially connected first detection sub-networks to mine features in each level (context), the detection result of the previous first detection sub-network is used to guide the current first detection sub-network to obtain a detection result from the edge enhancement feature map corresponding thereto, so as to enhance the detection of the tampered region. The first detection sub-network can be any neural network capable of detecting tampered regions in related technologies.

[0121] Then, in step 46, the tamper detection model is trained based on each second segmentation map and first label data.

[0122] The first label data can include a label segmentation map generated based on the first sample image. At this time, the electronic device can construct a total loss based on the difference between each second segmentation map and the label segmentation map, and then adjust the model parameters of the tamper detection model to train the tamper detection model with the goal of minimizing the total loss, i.e., minimizing the difference between each second segmentation map and the label segmentation map.

[0123] In yet some examples, the first label data includes a label segmentation map and a label edge map generated based on the first sample image. Correspondingly, the step 46 can include: training the tampering detection model based on the second tampering edge map, the second segmentation maps, and the label tampering edge map and the label segmentation map in the first label data. In this example, the electronic device can jointly determine a total loss based on a difference between the second tampering edge map and the label edge map, and a difference between the second segmentation maps and the label segmentation map, and then adjust model parameters of the tampering detection model to train the tampering detection model with a target of minimizing the total loss, i.e., minimizing the difference between the second tampering edge map and the label edge map, and the difference between the second segmentation maps and the label segmentation map.

[0124] In some possible examples, the aforementioned first label data includes a label tampering edge map and a label segmentation map generated based on the first sample image; correspondingly, the step 46 includes steps 461-464:

[0125] In the step 461, a first loss is constructed based on a difference between the second tampering edge map and the label tampering edge map. In some possible examples, the electronic device can construct the first loss based on a preset first loss function for measuring similarity between two objects, based on the difference between the second tampering edge map and the label tampering edge map, where the first loss is negatively related to the similarity between the second tampering edge map and the label tampering edge map, and correspondingly, the higher the similarity between the second tampering edge map and the label tampering edge map, the smaller the first loss. For example, the first loss function is, for example, a dice loss function.

[0126] In the step 462, a second loss is constructed based on a difference between the label segmentation map and each of the second segmentation maps. In this step, the electronic device can use a weighted binary cross-entropy loss function and / or a weighted IOU (Intersection over Union) loss function to construct a sub-loss corresponding to each of the second segmentation maps based on a difference between the label segmentation map and each of the second segmentation maps, and then determine the second loss based on the sub-losses corresponding to the second segmentation maps. For example, a sum of the sub-losses corresponding to the second segmentation maps is determined as the second loss, or an average of the sub-losses corresponding to the second segmentation maps is determined as the second loss.

[0127] In the step 463, a total loss is constructed, where the total loss is positively related to the first loss and the second loss. Then, in the step 464, the tampering detection model is trained with a target of minimizing the total loss.

[0128] Specifically, the electronic device can determine a sum of a product of the first loss and a weight value corresponding to the first loss and a product of the second loss and a weight value corresponding to the second loss as a total loss; and then, adjust model parameters of the main branch network, the reversible branch network, the first edge detection network, and the first region detection network in the tampering detection model to train the tampering detection model, with a goal of minimizing the total loss, i.e., minimizing a difference between the second tampering edge map and the label tampering edge map and minimizing a difference between the label segmentation map and each of the second segmentation maps.

[0129] The steps 41-46 above are a model iteration training process for the tampering detection model. To train a better tampering detection model, the above process can be performed multiple times. That is, after step 46, based on the updated model parameters of the tampering detection model, return to perform step 41.

[0130] The stop condition of the above model iteration training process can include that the number of iteration training reaches a preset number threshold, or the iteration training duration reaches a preset duration, or the total loss is less than a set loss threshold, and the like.

[0131] In the above process, the reversible branch network can alleviate the information bottleneck problem caused by the excessive depth of the deep learning-based neural network, that is, the depth of the main branch network does not need to be too deep (i.e., the number of first extraction layers in the main branch network does not need to be too large), and reliable gradient information can be generated with the assistance of the reversible branch network. Then, based on such gradient information, the main branch network can learn more useful features for the tampering region detection task, and the information bottleneck problem caused by the excessive depth of the deep learning-based neural network can be solved.

[0132] Moreover, for the tampering detection model, the feature maps corresponding to the second extraction layers of different layers in the reversible branch network can be used to perform different detection tasks, such as detecting tampering regions of different sizes, respectively. For example, the second extraction layers with a smaller number of layers can be guided to learn features of smaller tampering regions in an image, and the second extraction layers with a larger number of layers can be guided to learn features of larger tampering regions in an image. Then, when constructing the total loss, the second segmentation maps corresponding to the multiple second extraction layers output by the first region detection network are combined, so that the total loss aggregates the information of the tampering regions of the multiple second extraction layers in the reversible branch network. Then, based on the gradient information obtained from the total loss, the gradient information of the tampering regions corresponding to the multiple second extraction layers in the reversible branch network is aggregated, and is subsequently back-propagated to the main branch network. As a result, the main branch network can learn features of tampering regions of different sizes, and can preserve more local detail information. Therefore, the tampering detection model can detect tampering regions of different sizes, and the accuracy of the tampering region detection result of the tampering detection model can be improved.

[0133] In yet some possible examples, the first label data can further include a label value indicating whether the first sample image contains a tampered region; as shown in FIG. 8, the tampering detection model includes a main branch network, a reversible branch network, a first edge detection network, a first region detection network, and a tampered region classification network, wherein the main branch network includes a plurality of first extraction layers connected in series, and the reversible branch network includes a plurality of second extraction layers connected in series.

[0134] In some possible examples, based on the structure shown in FIG. 8, in the training process of the tampering detection model, after the electronic device obtains each second segmentation image through the foregoing steps 41-45, in step 46, the electronic device can include the following steps 51-52: in step 51, the tampered region classification network is used to determine, based on a target segmentation image in each second segmentation image, a predicted value indicating whether the first sample image contains a tampered region. The target segmentation image can be all the second segmentation images, or a segmentation image output by the last first detection sub-network in each second segmentation image.

[0135] For example, the tampered region classification network can include at least a classifier (such as a multilayer perceptron or other classifier or a fully connected layer), so as to determine, through the classifier, based on the target segmentation image, the predicted value indicating whether the first sample image contains a tampered region.

[0136] Then, in step 52, the tampering detection model is trained based on the difference between the label value and the predicted value. In this step, a total loss is constructed based on the difference between the label value and the predicted value, and the model parameters of the tampering detection model are adjusted to minimize the total loss, i.e., to minimize the difference between the label value and the predicted value, so as to train the tampering detection model.

[0137] In some possible examples, based on the structure shown in FIG. 8, in the training process of the tampering detection model, after the electronic device obtains each second segmentation image and each second tampered edge image through the foregoing steps 41-46, in step 46, the electronic device can include the following steps 61-62: in step 61, the tampered region classification network is used to determine, based on a target segmentation image in each second segmentation image and the second tampered edge image, a predicted value indicating whether the first sample image contains a tampered region. Then, in step 62, the tampering detection model is trained based on the difference between the label value and the predicted value.

[0138] In an implementation, the tampered region classification network can include two feature extraction layers a arranged in parallel, a feature fusion layer connected to the two feature extraction layers a, and a classifier connected to the feature fusion layer. Accordingly, the electronic device can input the target segmentation image into one feature extraction layer a1, and input the second tampered edge map into the other feature extraction layer a2, and then input the outputs of the two feature extraction layers into the feature fusion layer to obtain a fused feature map, and process the fused feature map by using the classifier to obtain a predicted value. Then, the tampered detection model is trained based on the difference between the label value and the predicted value. The feature extraction layer can be implemented based on any network capable of feature extraction in the related art. In some implementations, the feature extraction layer can include a GeM pooling layer (Generalized Mean pooling layer) so as to extract more features beneficial to the tampered region detection task from the target segmentation image and the second tampered edge map, and improve the accuracy of the predicted value. The feature fusion layer can fuse the two inputs by means of pixel-by-pixel addition, pixel-by-pixel multiplication or splicing, etc.

[0139] In yet another implementation, the tampered region classification network can include a first classification sub-network and a second classification sub-network. The electronic device can process the target segmentation image by using the first classification sub-network to obtain a first value, process the second tampered edge map by using the second classification sub-network to obtain a second value, perform weighted summation on the first value and the second value to obtain a predicted value, and then train the tampered detection model based on the difference between the label value and the predicted value. Each classification sub-network can include at least a classifier, and can further include a feature extraction layer before the classifier.

[0140] In yet another implementation, the tampered region classification network can further include a third classification sub-network. The electronic device can further process the feature map of the last second extraction layer in the plurality of third feature maps by using the third classification sub-network to obtain a third value, and then perform weighted summation on the first value, the second value and the third value to obtain a predicted value, and then train the tampered detection model based on the difference between the label value and the predicted value. This can comprehensively consider the multi-path information decision mechanism to better improve the tampered detection accuracy of the tampered detection model. The structure of the third classification sub-network can refer to the structures of the first classification sub-network and the second classification sub-network.

[0141] In yet another implementation, the main branch network, the reversible branch network, the first edge detection network and the first region detection network in the training of the tampered detection model can be trained based on the aforementioned sample images and their label data, and the model parameters of the main branch network, the reversible branch network, the first edge detection network and the first region detection network can be fixed during the training of the tampered region classification network.

[0142] In some examples, the training method of the tampering detection model in another embodiment of the present specification. As shown in FIG. 9, the tampering detection model includes a main branch network, a reversible branch network, a first edge detection network, a first region detection network, a second edge detection network and a second region detection network, the main branch network includes a plurality of first extraction layers connected in series, and the reversible branch network includes a plurality of second extraction layers connected in series. In the training process of the tampering detection model, the method includes steps 71-78 as follows:

[0143] In step 71, a first sample image and its first label data are obtained, wherein the first sample image is a card image containing a tampered region.

[0144] In step 72, feature extraction is performed on the first sample image based on the main branch network to obtain a plurality of second feature maps corresponding to a plurality of specified first extraction layers.

[0145] In step 73, the first sample image is processed based on the reversible branch network to obtain a plurality of third feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes: through a third extraction layer in the plurality of second extraction layers, performing feature extraction on the third feature map corresponding to the previous layer and the set second feature map corresponding thereto to obtain its output feature map, and based on the output feature map and the second feature map of the first extraction layer of the same layer, obtaining the corresponding third feature map.

[0146] In step 74, a second tampering edge map is obtained based on the plurality of third feature maps through the first edge detection network.

[0147] Wherein, the implementation principle of steps 71-74 is similar to the implementation principle of the aforementioned steps 41-44, and the implementation process can be referred to the implementation process of steps 41-44, which will not be repeated here.

[0148] In step 75, a third tampering edge map is obtained based on the plurality of second feature maps through the second edge detection network. As shown in FIG. 9, the structure of the second edge detection network is similar to that of the first edge detection network, and the specific structure can be referred to the specific structure of the first edge detection network. The implementation principle of step 75 is similar to the implementation principle of the aforementioned step 74, and the implementation process can be referred to the implementation process of step 74, which will not be repeated here.

[0149] In step 76, a plurality of second segmentation maps are obtained based on the plurality of third feature maps and the second tampering edge map through the first region detection network, wherein the second segmentation map includes a predicted tampered region in the first sample image. The implementation principle of step 76 is similar to the implementation principle of the aforementioned step 45, and the implementation process can be referred to the implementation process of step 45, which will not be repeated here.

[0150] At step 77, a plurality of third segmentation maps are obtained based on the third tampered edge map and the plurality of second feature maps by a second region detection network. The structure of the second region detection network is similar to that of the first region detection network. As shown in FIG. 9, the second edge detection network includes a plurality of second detection sub-networks (including “CAM4”, “CAM5” and “CAM6” as shown in FIG. 9) connected in series, and a plurality of second edge enhancement sub-networks (including “EFM5” corresponding to the extraction layer 12, with the second feature map f12 corresponding to the extraction layer 12 as the input; “EFM6” corresponding to the extraction layer 13, with the second feature map f13 corresponding to the extraction layer 13 as the input; “EFM7” corresponding to the extraction layer 14, with the second feature map f14 corresponding to the extraction layer 14 as the input; and “EFM8” corresponding to the extraction layer 15, with the second feature map f15 corresponding to the extraction layer 15 as the input) corresponding to each designated first extraction layer. The structure of the second detection sub-network and the second edge enhancement sub-network is similar to that of the first detection sub-network and the first edge enhancement sub-network in the first region detection network, respectively. The specific structure can be referred to the specific structure of the first detection sub-network and the first edge enhancement sub-network, respectively. The implementation principle of step 77 is similar to that of step 76. The implementation process can be referred to the implementation process of step 76, which is not repeated here.

[0151] At step 78, the tampering detection model is trained based on the second segmentation maps, the third segmentation maps and the first label data.

[0152] For example, the first label data includes a label segmentation map generated based on the first sample image. Accordingly, the electronic device can construct a sub-loss corresponding to each segmentation map (including the second segmentation map and the third segmentation map) based on the difference between the label segmentation map and each second segmentation map and each third segmentation map. Then, the total loss is determined by combining the sub-losses. The total loss is positively correlated with the sub-losses. Then, the tampering detection model is trained to minimize the total loss, i.e., to minimize each sub-loss.

[0153] For another example, the first label data includes a label edge map and a label segmentation map generated based on the first sample image. Accordingly, at step 78, the electronic device can construct a first loss based on a difference between the second tampered edge map and the label tampered edge map, construct a second loss based on a difference between the label segmentation map and each of the second segmentation maps, construct a third loss based on a difference between the third tampered edge map and the label tampered edge map, construct a fourth loss based on a difference between the label segmentation map and each of the third segmentation maps, and construct a total loss by combining the first loss, the second loss, the third loss, and the fourth loss, where the total loss is positively correlated with the first loss, the second loss, the third loss, and the fourth loss, respectively. Then, the model parameters of the main branch network, the reversible branch network, the first edge detection network, the first region detection network, the second edge detection network, and the second region detection network in the tamper detection model are adjusted to train the tamper detection model with the goal of minimizing the total loss.

[0154] The specific determination process of the third loss and the fourth loss can refer to the specific determination process of the first loss and the second loss described above, which will not be repeated here.

[0155] In the above process, the feature maps corresponding to the second extraction layers of different layers in the reversible branch network of the tamper detection model can be used to perform different detection tasks, and the feature maps corresponding to the first extraction layers of different layers in the main branch network can be used to perform different detection tasks, for example, different sizes of tampered regions can be detected respectively. The second extraction layer (the first extraction layer) with a smaller number of layers can be guided to learn the features of smaller tampered regions in the image, and the second extraction layer (the first extraction layer) with a larger number of layers can be guided to learn the features of larger tampered regions in the image. Subsequently, when constructing the total loss, the second segmentation maps corresponding to the multiple second extraction layers output by the first region detection network and the third segmentation maps corresponding to the multiple first extraction layers output by the second region detection network are combined, so that the total loss aggregates the information of the tampered regions of the multiple second extraction layers in the reversible branch network and the information of the tampered regions of the multiple first extraction layers in the main branch network. Then, based on the gradient information obtained from the total loss, the gradient information of the tampered regions corresponding to the multiple second extraction layers in the reversible branch network and the gradient information of the tampered regions of the multiple first extraction layers in the main branch network are aggregated, and then are reversely transmitted to the main branch network. This can make the main branch network not need to set more first extraction layers, and still can better learn the features of tampered regions of various sizes, and can preserve more local detail information. Therefore, the tamper detection model can better improve the accuracy of the tampered region detection result of the tamper detection model, and solve the information bottleneck problem caused by the excessive depth of the neural network based on deep learning.

[0156] In some possible examples, the tampering detection model can further include the aforementioned first classification network. Accordingly, the tampering detection model can be trained jointly with respect to the aforementioned parts (e.g., the aforementioned main branch network, the reversible branch network, the first edge detection network, the first region detection network, etc.) and the first classification network. In this case, the label data corresponding to each sample image in the training data used for training the tampering detection model can further include position information (e.g., in the form of a bounding box, referred to as label position information) of the region where each sample key object is located in the sample image, and label data indicating whether the region where each sample key object is located is a tampered region. When constructing a loss function for adjusting the tampering detection model, in addition to being constructed based on the aforementioned losses used for training the tampering detection model, the loss function needs to be constructed based on the difference between the label data indicating whether the region where each sample key object is located is a tampered region and the prediction result data output by the first classification network indicating whether the region where each sample key object is located is a tampered region.

[0157] Specifically, the tampering detection model can further include a first classification network. The aforementioned first label data can include a label map (which can include the aforementioned label segmentation map and can further include the aforementioned label edge map) generated based on the first sample image, and label position information of the region where each sample key object is located in the first sample image, and label data indicating whether the region where each sample key object is located is a tampered region.

[0158] Based on the training process of the tampering detection model shown in steps 41-45, the training process of the tampering detection model can further include steps 81-82 as follows:

[0159] In step 81, based on the aforementioned multiple second feature maps and the label position information, a sample region feature map corresponding to the region where each sample key object is located is determined. The implementation principle of step 81 is similar to that of step S330, and the implementation process can refer to the implementation process of step S330, which will not be repeated here.

[0160] Then, in step 82, the first classification network is used to obtain prediction result data indicating whether the region where each sample key object is located is a tampered region based on the sample region feature map. The implementation principle of step 82 is similar to that of step S340, and the implementation process can refer to the implementation process of step S340, which will not be repeated here.

[0161] In step 46, the tampering detection model is trained based on the second segmentation map and the label map, the prediction result data, and the label data indicating whether the region where each sample key object is located is a tampered region.

[0162] In this step, the electronic device can construct a sub-loss corresponding to each second segmentation map based on the difference between each second segmentation map and the label map. The process of constructing the sub-loss corresponding to each second segmentation map based on the difference between each second segmentation map and the label segmentation map has been described above and will not be repeated here.

[0163] The electronic device can use a cross-entropy loss function to construct a region loss based on the difference between the predicted result data and the label data indicating whether the region where the key object of each sample is located is a tampered region. Then, based on the region loss and the sub-loss corresponding to each second segmentation map, the total loss is jointly determined. The total loss is positively correlated with the sub-loss corresponding to each second segmentation map and the region loss. Then, the model parameters of each part of the tampering detection model are adjusted to minimize the total loss.

[0164] It can be understood that the main branch network, the reversible branch network, the first edge detection network, the first region detection network, the second edge detection network, the second region detection network, the tampered region classification network, and the first classification network in the tampering detection model can also be trained to better improve the accuracy of the detection result of the tampering detection model. The corresponding training process can refer to the aforementioned training process and will not be repeated here.

[0165] After training the tampering detection model through the above-mentioned manner, the position information of the target region in the card image can be jointly determined based on the main branch network, the first edge detection network, the first region detection network, and the first classification network in the tampering detection model to realize the detection of whether the card image contains a tampered region. In the tampering detection process, as shown in FIG. 4, the method includes the following steps S410-S450:

[0166] In step S410, the position information of the target region in the card image to be detected is obtained, wherein the target region is the region where the key object in the card image is located.

[0167] In step S420, one or more first feature maps are obtained by performing feature extraction on the card image through one or more first extraction layers. The one or more first extraction layers are the serially connected first extraction layers in the main branch network. The one or more first feature maps are the plurality of first feature maps obtained by the plurality of designated first extraction layers in the main branch network based on the card image.

[0168] At step S430, a region feature map corresponding to the target region is determined based on the one or more first feature maps and the position information. For example, the region feature map corresponding to the target region can be determined based on a specified first feature map in the one or more first feature maps and the position information of the target region. For example, the specified first feature map can be the first feature map fa3 output by the extraction layer 13 in the middle position of the main branch network, as shown in FIG. 10. For another example, the region feature map corresponding to the target region can be determined based on at least two specified first feature maps in the one or more first feature maps and the position information of the target region. For example, the at least two specified first feature maps can include the first feature map fa3 output by the extraction layer 13 and the first feature map output by the extraction layer 14, as shown in FIG. 10.

[0169] At step S440, a first detection result of whether the target region is a tampered region is obtained based on the region feature map by the first classification network. For example, the electronic device can process the region feature map by using a specified region alignment algorithm (e.g., the “region alignment algorithm” shown in FIG. 10) to obtain a feature map of a second size corresponding to the region feature map, and then obtain the first detection result of whether the target region is a tampered region based on the feature map of the second size corresponding to the region feature map by the first classification network.

[0170] The implementation principle of steps S410-S440 is similar to that of steps S310-S340 shown in FIG. 3, and the implementation process can be referred to the implementation process of steps S310-S340, which will not be repeated here.

[0171] At step S450, a first tampered edge map is obtained based on the one or more first feature maps by the first edge detection network. The implementation principle of step S450 is similar to that of the aforementioned step 44, and the implementation process can be referred to the implementation process of the aforementioned step 44, which will not be repeated here. As shown in FIG. 10, the first tampered edge map can be obtained based on the feature map corresponding to the target first extraction layer (e.g., the feature map fa2 shown in FIG. 10) and the feature map corresponding to the last layer of the first extraction layer (e.g., the feature map fa5 shown in FIG. 10) in the plurality of first feature maps by the first edge detection network, where the number of layers of the target first extraction layer in the main branch network is not higher than the preset number of layers.

[0172] At step S460, a first segmentation map is obtained based on the first tampered edge map and the one or more first feature maps by the first region detection network, where the first segmentation map indicates a region suspected to be tampered predicted by the tampering detection model. The implementation principle of step S460 is similar to that of the aforementioned step 45, and the implementation process can be referred to the implementation process of the aforementioned step 45, which will not be repeated here.

[0173] In some possible examples, the first region detection network comprises a plurality of serially connected first detection sub-networks, and the first segmentation map can be a segmentation map output by a last first detection sub-network.

[0174] At step S470, a tampering detection result of whether the card image contains a tampered region is determined based on at least the first detection result and the first segmentation map.

[0175] It can be understood that the step flow shown in FIG. 4 is only a schematic diagram of the tampering detection process, and does not constitute a limitation on the execution order between the steps. In some implementations, steps S450-S460 can be executed before S430-S440, and can also be executed in parallel with S430-S440, etc.

[0176] In some possible examples, the electronic device can determine whether the card image contains a tampered region from a global perspective of the card image based on whether the first segmentation map contains an image region with a pixel value of the first value, and then combine the first detection result determined from a local detailed region perspective of the card image to determine the tampering detection result of whether the card image contains a tampered region, so as to better ensure the accuracy of the determined tampering detection result. For example, if the first segmentation map contains an image region with a pixel value of the first value, and the number of pixel values of the first value in the image region is not less than a preset number, it can be determined from a global perspective of the card image that the card image contains a tampered region; otherwise, it is determined from a global perspective of the card image that the card image does not contain a tampered region.

[0177] In another implementation, if the first detection result indicates that the target region is a tampered region and / or it is determined from a global perspective of the card image that the card image contains a tampered region, it can be determined that the tampering detection result is that the card image contains a tampered region; if the first detection result indicates that the target region is not a tampered region, and it is determined from a global perspective of the card image that the card image does not contain a tampered region, it is determined that the tampering detection result is that the card image does not contain a tampered region, which can improve the detection rate of the tampered region in the card image, and better ensure the security of the online verification system.

[0178] In some possible examples, the tampering detection model obtained by the foregoing training further comprises a tampered region classification network, and at step S470, steps 91-92 can be included.

[0179] At step 91, a second detection result is determined based on at least the first segmentation map by using the tampered region classification network in the tampering detection model. The process of obtaining the second detection result indicating whether the card image contains a tampered region based on the first segmentation map by using the tampered region classification network can refer to the implementation process of step 51 described above, and will not be repeated here.

[0180] In some possible examples, at step 91, the electronic device can determine the second detection result based on the first tampered edge map and the first segmentation map by using the tampered region classification network. Details can be referred to the implementation process of step 61 in the foregoing embodiments, which are not described herein again.

[0181] After obtaining the second detection result, at step 92, the electronic device can determine the tampered detection result based on the first detection result and the second detection result.

[0182] In some examples, the second detection result can be a score value indicating whether the card image contains a tampered region, and the first detection result can be a score value indicating whether the corresponding target region is a tampered region. In some implementations, the process of determining the tampered detection result can be: if at least one of the first detection result and the second detection result is greater than a preset threshold, it can be determined that the tampered detection result indicates that the card image contains a tampered region; otherwise, it is determined that the tampered detection result indicates that the card image does not contain a tampered region. Alternatively, in the case where the first detection result is multiple, the maximum value of the first detection result can be determined from the multiple first detection results. The maximum value of the first detection result and the second detection result are weighted and summed to obtain a final detection result, which includes a final score value obtained by weighted sum of the maximum value of the first detection result and the second detection result. If the final score value is greater than the preset threshold, it is determined that the tampered detection result indicates that the card image contains a tampered region; otherwise, it is determined that the tampered detection result indicates that the card image does not contain a tampered region, so as to better improve the accuracy of the determined tampered detection result.

[0183] The above process combines the detection result of the local key object region in the card image and the detection result of the global angle of the card image to jointly determine the tampered detection result, which can not only improve the accuracy of the tampered detection result, but also improve the detection rate of the small tampered region in the card image.

[0184] In yet some possible examples, in order to obtain a tampered detection result indicating whether the card image contains a tampered region with higher accuracy, the electronic device can jointly determine the second detection result by using the multiple first feature maps containing global features, the first tampered edge map containing edge features, and the first segmentation map containing pixel features. For example, the tampered region classification network can include a first classification subnetwork, a second classification subnetwork, and a third classification subnetwork. Correspondingly, at step S470, the process can include steps 101-104.

[0185] At step 101, the first result is determined by the third classification subnetwork based on the fourth feature map corresponding to the last layer of the main branch network in the plurality of first feature maps.

[0186] In some examples, the electronic device can determine the fourth feature map corresponding to the last layer of the main branch network from the plurality of first feature maps, and then input the fourth feature map into the third classification subnetwork to obtain the first result, which is used to indicate whether the card image contains a tampered region. In some possible examples, the first result includes a first subvalue indicating that the card image contains a tampered region, wherein the higher the first subvalue, the greater the possibility that the card image contains a tampered region.

[0187] As shown in FIG. 10, the third classification subnetwork can be arranged after the main branch network, and the input of the third classification subnetwork can be the output corresponding to the last layer of the main branch network.

[0188] In yet other examples, the electronic device can also determine the first result based on the fourth feature map and at least one other feature map in the plurality of first feature maps during the determination of the first result. Specifically, the electronic device can input the fourth feature map and the at least one other feature map in the plurality of first feature maps into the third classification subnetwork to fuse the fourth feature map and the at least one other feature map in the plurality of first feature maps by the third classification subnetwork, obtain a fused feature, and then process the fused feature by the third classification subnetwork to obtain the first result.

[0189] At step 102, the second result is determined by the second classification subnetwork based on the first tampered edge map.

[0190] In some examples, the electronic device can determine the second result by the second classification subnetwork based on the first tampered edge map, and the second result is used to indicate whether the card image contains a tampered region. For example, the second result includes a second subvalue indicating whether the card image contains a tampered region, wherein the higher the second subvalue, the greater the possibility that the card image contains a tampered region.

[0191] As shown in FIG. 10, the second classification subnetwork can be arranged after the first edge detection network, and the input of the second classification subnetwork can be the output (e.g., the first tampered edge map) corresponding to the first edge detection network.

[0192] At step 103, the third result is determined by the first classification subnetwork based on the first segmentation map. For example, the third result includes a third subvalue indicating whether the card image contains a tampered region, wherein the higher the third subvalue, the greater the possibility that the card image contains a tampered region.

[0193] As shown in FIG. 10, the first classification sub-network can be arranged after the first region detection network, and the input of the first classification sub-network is the output corresponding to each first detection sub-network in the first region detection network, or the output corresponding to the last first detection sub-network in the first region detection network.

[0194] In step 104, the tampering detection result is determined by combining the first result, the second result and the third result, and the first detection result. The implementation principle of combining the first result, the second result and the third result, and the first detection result to determine the tampering detection result is similar to the implementation principle of determining the tampering detection result based on the first detection result and the second detection result as described above. The implementation process can refer to the process of determining the tampering detection result based on the first detection result and the second detection result as described above, and will not be described here.

[0195] In some possible examples, in the process of handling a designated business online, in a scenario where the user needs to verify his / her identity by using his / her card, the user can upload an image containing the image of the card, i.e., the card image to be detected. After obtaining the card image, the electronic device needs to perform tampering detection on the card image by using the tampering detection method provided in this embodiment, i.e., to detect whether the card image contains a tampered region. If it is determined that the card image does not contain a tampered region, the subsequent related handling process of the designated business can be continued. If it is determined that the card image contains a tampered region, the electronic device can send a prompt information and stop the subsequent related handling process of the designated business.

[0196] It can be understood that, in the process of training the tampering detection model, the reversible branch network is used to assist the main branch network to learn more information in the image, and to ensure the accuracy of the detection result. Correspondingly, in the process of tampering detection, in order to improve the detection efficiency, the main branch network, the first edge detection network and the first region detection network in the tampering detection model can be used to jointly perform the tampering detection process, and an accurate detection result can be obtained. In addition, in the above process, the edge features in the image to be detected are enhanced based on the first edge detection network and the first region detection network, and a multi-path information joint decision mechanism is introduced, so that the tampering region detection accuracy is better improved.

[0197] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments, and the desired results can still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be implemented in the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0198] Corresponding to the above method embodiments, the specification embodiments provide a tampering detection apparatus 1100, a schematic block diagram of which is shown in FIG. 11, comprising: a first acquisition module 1110 configured to acquire position information of a target region in a card image to be detected, wherein the target region is a region where a key object in the card image is located; a first obtaining module 1120 configured to perform feature extraction on the card image through one or more first extraction layers to obtain one or more first feature maps; a first determination module 1130 configured to determine a region feature map corresponding to the target region based on the one or more first feature maps and the position information; a second obtaining module 1140 configured to obtain a first detection result of whether the target region is a tampered region based on the region feature map through a first classification network; and a second determination module 1150 configured to determine a tampering detection result of whether the card image contains a tampered region based at least on the first detection result.

[0199] In some possible examples, the region where the key object is located includes a region where a specified string is located and / or a region where a face image is located.

[0200] In some possible examples, the first acquisition module 1110 is specifically configured to perform detection on the card image based on a preset detection algorithm to obtain position information of each candidate target region in the card image; obtain a feature map of each candidate target region mapped to a first size based on the position information of each candidate target region using a specified region alignment algorithm; determine a classification result of whether each candidate target region is the target region based on the feature map of each candidate target region mapped to the first size through a second classification network; and determine the position information of the target region from the position information of each candidate target region based on the classification result.

[0201] In some possible examples, the one or more first extraction layers belong to a tampering detection model; and the tampering detection model further comprises a first edge detection network and a first region detection network.

[0202] The apparatus further comprises: a third obtaining module (not shown in the figure) configured to obtain a first tampering edge map based on the one or more first feature maps through the first edge detection network; and a fourth obtaining module (not shown in the figure) configured to obtain a first segmentation map based on the first tampering edge map and the one or more first feature maps through the first region detection network, wherein the first segmentation map indicates a region suspected of being tampered predicted by the tampering detection model.

[0203] The second determining module 1150 is specifically configured to determine a tampering detection result of whether the card image contains a tampered region based on at least the first detection result and the first segmentation map.

[0204] In some possible examples, the second determining module 1150 is specifically configured to determine a second detection result based on at least the first segmentation map by using a tampered region classification network in the tampering detection model.

[0205] The tampering detection result is determined based on the first detection result and the second detection result.

[0206] In some possible examples, the second obtaining module 1140 is specifically configured to process the region feature map by using a specified region alignment algorithm to obtain a feature map of a second size corresponding to the region feature map.

[0207] The first classification network is used to obtain a first detection result of whether the target region is a tampered region based on the feature map of the second size corresponding to the region feature map.

[0208] In some possible examples, the first feature map is a plurality of first feature maps.

[0209] The first determining module 1130 is specifically configured to obtain a fusion feature map by fusing at least two specified first feature maps in the plurality of first feature maps, and determine a region feature map corresponding to the target region from the fusion feature map based on the position information and a size ratio between the fusion feature map and the card image.

[0210] In some possible examples, the tampering detection model further includes a reversible branch network; the reversible branch network includes a plurality of second extraction layers connected in series; and the apparatus further includes a second obtaining module (not shown in the figure) configured to obtain a first sample image and first label data thereof, where the first label data is used to indicate whether the first sample image contains a tampered region.

[0211] A fifth obtaining module (not shown in the figure) is configured to perform feature extraction on the first sample image based on a plurality of first extraction layers connected in series in the main branch network to obtain a plurality of second feature maps corresponding to a plurality of specified first extraction layers.

[0212] The processing module (not shown in the figure) is configured to process the first sample image based on the reversible branch network to obtain a plurality of third feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes: performing feature extraction on the third feature map corresponding to the previous layer and the set second feature map corresponding thereto by a third extraction layer in the plurality of second extraction layers to obtain the output feature map thereof, and obtaining the corresponding third feature map based on the output feature map and the second feature map of the first extraction layer in the same layer, the third extraction layer being a specified layer in the plurality of second extraction layers starting from the second second extraction layer.

[0213] The sixth obtaining module (not shown in the figure) is configured to obtain a second tampering edge map based on the plurality of third feature maps by the first edge detection network.

[0214] The seventh obtaining module (not shown in the figure) is configured to obtain a plurality of second segmentation maps based on the plurality of third feature maps and the second tampering edge map by the first region detection network.

[0215] The training module (not shown in the figure) is configured to train the tampering detection model based on each second segmentation map and the first label data.

[0216] In some possible examples, the tampering detection model further includes the first classification network; the first label data includes a label map generated based on the first sample image, label position information of the region where each sample key object is located in the first sample image, and label data indicating whether the region where each sample key object is located is a tampering region;

[0217] The device further includes a third determining module (not shown in the figure) configured to determine a sample region feature map corresponding to the region where each sample key object is located based on the plurality of second feature maps and the label position information.

[0218] The eighth obtaining module (not shown in the figure) is configured to obtain the prediction result data of whether the region where each sample key object is located is a tampering region based on each sample region feature map by the first classification network.

[0219] The training module is specifically configured to train the tampering detection model based on each second segmentation map and the label map, the prediction result data, and the label data indicating whether the region where each sample key object is located is a tampering region.

[0220] In some possible examples, the first label data includes a label tampering edge map and a label segmentation map generated based on the first sample image; and the training module is specifically configured to train the tampering detection model based on the second tampering edge map, the second segmentation maps, and the label tampering edge map and the label segmentation map in the first label data.

[0221] In some possible examples, the first label data includes a label value used to indicate whether the first sample image contains a tampering region; and the tampering detection model further includes a tampering region classification network.

[0222] The training module is specifically configured to determine, by using the tampering region classification network, a predicted value used to indicate whether the first sample image contains a tampering region based on a target segmentation image in the second segmentation images and the second tampering edge map; and train the tampering detection model based on a difference between the label value and the predicted value.

[0223] In some possible examples, the tampering detection model further includes a second edge detection network and a second region detection network, and the apparatus further includes: an eighth obtaining module (not shown in the figures) configured to obtain, by using the second edge detection network, a third tampering edge map based on the second feature maps; and a ninth obtaining module (not shown in the figures) configured to obtain, by using the second region detection network, third segmentation maps based on the third tampering edge map and the second feature maps; and the training module is specifically configured to train the tampering detection model based on the second segmentation maps, the third segmentation maps, and the first label data.

[0224] The device embodiments correspond to the method embodiments, and specific descriptions can be referred to the descriptions of the method embodiments, which will not be repeated here. The device embodiments are based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. Specific descriptions can be referred to the corresponding method embodiments.

[0225] The present specification also provides a computer readable storage medium having a computer program stored thereon, which causes a computer to execute the tampering detection method provided in the present specification when the computer program is executed in the computer.

[0226] The present specification also provides a computing device including a memory and a processor, the memory having executable code stored therein, and the processor implementing the tampering detection method provided in the present specification when executing the executable code.

[0227] Each of the embodiments described in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, the storage medium and computing device embodiments are described simply because they are substantially similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0228] Those skilled in the art can realize that the functions described in the embodiments of the present application in one or more of the examples above can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, the functions can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium.

[0229] The above detailed description of the specific embodiments of the present application has further explained the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application shall be included in the protection scope of the present application.

Claims

1. A tamper detection method, comprising: obtaining position information of a target region in a card image to be detected, wherein the target region is a region where a key object in the card image is located; performing feature extraction on the card image through one or more first extraction layers to obtain one or more first feature maps; determining a region feature map corresponding to the target region based on the one or more first feature maps and the position information; obtaining a first detection result of whether the target region is a tampered region based on the region feature map through a first classification network; determining a tamper detection result of whether the card image contains a tampered region based on at least the first detection result.

2. The method of claim 1, wherein, The region where the key object is located includes a region where a specified string is located and / or a region where a face image is located.

3. The method of claim 1, wherein, The obtaining of the position information of the specified target region in the card image to be detected comprises: detecting the card image based on a preset detection algorithm to obtain position information of each candidate target region in the card image; obtaining a feature map of each candidate target region mapped to a first size based on the position information of each candidate target region using a specified region alignment algorithm; determining a classification result of whether each candidate target region is the target region based on the feature map of each candidate target region mapped to the first size through a second classification network; determining the position information of the target region from the position information of each candidate target region based on the classification result.

4. The method of claim 1, wherein, The one or more first extraction layers belong to a tamper detection model; the tamper detection model further comprises a first edge detection network and a first region detection network. The method further comprises: obtaining a first tamper edge map based on the one or more first feature maps through the first edge detection network; obtaining a first segmentation map based on the first tamper edge map and the one or more first feature maps through the first region detection network, wherein the first segmentation map indicates a region suspected of being tampered predicted by the tamper detection model; The determination of the tamper detection result of whether the card image contains a tampered region based on at least the first detection result comprises: determining the tamper detection result of whether the card image contains a tampered region based on at least the first detection result and the first segmentation map.

5. The method of claim 4, wherein, The determination of the tamper detection result of whether the card image contains a tampered region comprises: determining a second detection result based on at least the first segmentation map using a tampered region classification network in the tamper detection model; determining the tamper detection result based on the first detection result and the second detection result. The obtaining of the first detection result of whether the target region is a tampered region based on the region feature map through the first classification network comprises:

6. The method of claim 1, wherein, processing the region feature map to obtain a feature map of the region feature map mapped to a second size using a specified region alignment algorithm; obtaining the first detection result of whether the target region is a tampered region based on the feature map of the region feature map mapped to the second size through the first classification network. ​ 7. The method of claim 1, wherein, The first feature map is multiple; The determining the region feature map corresponding to the target region comprises: Fusing at least two specified first feature maps in the multiple first feature maps to obtain a fused feature map; Based on the position information and the size ratio between the fused feature map and the card image, the region feature map corresponding to the target region is determined from the fused feature map.

8. The method of claim 4, wherein, The tampering detection model further comprises a reversible branch network; the reversible branch network comprises a plurality of second extraction layers connected in series; The method further comprises: Obtaining a first sample image and first label data thereof, wherein the first label data is used to indicate whether the first sample image contains a tampered region; Based on a plurality of first extraction layers connected in series in the main branch network, feature extraction is performed on the first sample image to obtain a plurality of second feature maps corresponding to a plurality of specified first extraction layers; Based on the reversible branch network, the first sample image is processed to obtain a plurality of third feature maps corresponding to a plurality of specified second extraction layers, wherein the processing comprises: through a third extraction layer in the plurality of second extraction layers, feature extraction is performed on the third feature map corresponding to the previous layer and the set second feature map corresponding thereto to obtain its output feature map, based on the output feature map and the second feature map of the first extraction layer of the same layer, the corresponding third feature map is obtained, and the third extraction layer is a specified layer in the plurality of second extraction layers starting from the second second extraction layer; Based on the plurality of third feature maps, a second tampering edge map is obtained through the first edge detection network; Based on the plurality of third feature maps and the second tampering edge map, a plurality of second segmentation maps are obtained through the first region detection network; Based on each second segmentation map and the first label data, the tampering detection model is trained.

9. The method of claim 8, wherein, The tampering detection model further comprises the first classification network; the first label data comprises a label map generated based on the first sample image, label position information of a region where each sample key object is located, and label data indicating whether the region where each sample key object is located is a tampered region; The method further comprises: Based on the plurality of second feature maps and the label position information, a sample region feature map corresponding to the region where each sample key object is located is determined; Based on each sample region feature map, a prediction result data of whether the region where each sample key object is located is a tampered region is obtained through the first classification network; The training of the tampering detection model comprises: Based on each second segmentation map and the label map, and the prediction result data and the label data indicating whether the region where each sample key object is located is a tampered region, the tampering detection model is trained.

10. The method of claim 8, wherein, The first label data comprises a label tampering edge map and a label segmentation map generated based on the first sample image; The training of the tampering detection model comprises: Based on the second tampering edge map, each second segmentation map, and the label tampering edge map and the label segmentation map in the first label data, the tampering detection model is trained. 11.The method of claim 8, wherein the first label data comprises a label value indicating whether the first sample image contains a tampered region. The tampering detection model further comprises a tampering region classification network; The training of the tamper detection model comprises: using the tamper region classification network, determining a prediction value indicating whether the first sample image contains a tampered region based on a target second segmentation image in the second segmentation images and the second tamper edge map; training the tamper detection model based on a difference between the label value and the prediction value.

12. The method of claim 8, wherein the tamper detection model further comprises a second edge detection network and a second region detection network, and the method further comprises: obtaining, by the second edge detection network, a third tamper edge map based on the plurality of second feature maps; obtaining, by the second region detection network, a plurality of third segmentation images based on the third tamper edge map and the plurality of second feature maps; the training of the tamper detection model comprises: training the tamper detection model based on each second segmentation image, each third segmentation image, and the first label data.

13. A tamper detection apparatus, comprising: a first obtaining module configured to obtain position information of a target region in a card image to be detected, wherein the target region is a region where a key object in the card image is located; a first obtaining module configured to obtain one or more first feature maps by performing feature extraction on the card image through one or more first extraction layers; a first determining module configured to determine a region feature map corresponding to the target region based on the one or more first feature maps and the position information; a second obtaining module configured to obtain a first detection result of whether the target region is a tampered region based on the region feature map through a first classification network; a second determining module configured to determine a tamper detection result of whether the card image contains a tampered region based on at least the first detection result.

14. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and the processor executes the executable code to implement the method of any one of claims 1-12.

Images