Tamper detection method and device

Abstract

The embodiment of the invention provides a tampering detection method and device, and the method comprises the steps: obtaining the position information of a target region in a to-be-detected card image, and enabling the target region to be a region where a key object is located in the card image; performing feature extraction on the card image through one or more first extraction layers to obtain one or more first feature maps; based on the one or more first feature maps and the position information, determining a region feature map corresponding to the target region; obtaining a first detection result whether the target region is a tampered region or not based on the region feature map through a first classification network; and determining whether the card image contains a tampering detection result of the tampering area at least based on the first detection result.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a tampering detection method and device. Background Art

[0002] With the rapid development of the Internet, people can verify relevant cards and certificates (including bank cards, various types of identity documents, etc.) online through the online verification system provided by relevant institutions. Only after the verification is passed can they enjoy the online business services provided by relevant institutions.

[0003] At present, there are some behaviors of tampering with cards (bank cards, identity documents, etc.) (for example, physically covering the information of the card itself by using paper strips and / or other facial images, handwriting pollution, etc., or modifying the card information in the card image by image modification technology) in order to deceive the online verification system of relevant institutions. The above behaviors have greatly tested the security of the online verification system of relevant institutions. Therefore, how to provide a method for detecting tampered areas in cards has become an urgent problem to be solved. Summary of the invention

[0004] One or more embodiments of the present specification provide a tampering detection method and device to accurately detect the tampered area of ​​a card.

[0005] According to a first aspect, a tampering detection method is provided, comprising:

[0006] Acquire location information of a target area in the card image to be detected, wherein the target area is an area where a key object is located in the card image;

[0007] Extract features from the card image through one or more first extraction layers to obtain one or more first feature maps;

[0008] Determine a region feature map corresponding to the target region based on the one or more first feature maps and the location information;

[0009] Obtaining, by a first classification network, a first detection result of whether the target area is a tampered area based on the area feature map;

[0010] At least based on the first detection result, it is determined whether the card image contains a tampering detection result of a tampered area.

[0011] According to a second aspect, there is provided a tamper detection device, comprising:

[0012] A first acquisition module is configured to acquire position information of a target area in a card image to be detected, wherein the target area is an area where a key object is located in the card image;

[0013] A first obtaining module is configured to extract features from the card image through one or more first extraction layers to obtain one or more first feature maps;

[0014] A first determining module, configured to determine a regional feature map corresponding to the target area based on the one or more first feature maps and the location information;

[0015] A second obtaining module is configured to obtain a first detection result of whether the target area is a tampered area based on the area feature map through a first classification network;

[0016] The second determination module is configured to determine whether the card image contains a tampering detection result of a tampered area based at least on the first detection result.

[0017] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in the first aspect.

[0018] According to a fourth aspect, a computing device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in the first aspect is implemented.

[0019] According to the tampering detection method and device provided in the embodiment of this specification, the location information of the area where the key object of the target area in the card image to be detected is obtained; the card image is subjected to feature extraction through one or more first extraction layers to obtain one or more first feature maps; based on the one or more first feature maps and the location information, the regional feature map corresponding to the target area is determined; through the first classification network, based on the regional feature map, a first detection result of whether the target area is a tampered area is obtained; at least based on the first detection result, a tampering detection result of whether the card image contains a tampered area is determined. In the above process, for the target area in the card image (i.e., the area where the key object is likely to be tampered), based on its corresponding regional feature map, the first detection result of whether it is a tampered area is determined, which can achieve the focus and detection of the local key area in the card image, and then based on the first detection result of the local key area in the image, determine whether the card image contains a tampering detection result of the tampered area, and achieve the determination of the tampering detection result of the card image as a whole, and to a certain extent improve the accuracy of the tampering detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1A An exemplary card image including a tampered area provided for this specification;

[0022] Figure 1B An exemplary card image including a tampered area provided for this specification;

[0023] Figure 1C An exemplary card image including a tampered area provided for this specification;

[0024] Figure 2 A schematic diagram of an implementation framework of an embodiment disclosed in this specification;

[0025] Figure 3 A schematic diagram of a flow chart of a tampering detection method provided in an embodiment;

[0026] Figure 4 A schematic diagram of another process flow of the tampering detection method provided in the embodiment;

[0027] Figure 5 A schematic diagram of a structure of a tamper detection model provided in an embodiment;

[0028] Figure 6 A schematic diagram of a structure of a tamper detection model provided in an embodiment;

[0029] Figure 7 A schematic diagram of the structure of a first edge enhancement sub-network provided in an embodiment;

[0030] Figure 8 A schematic diagram of another structure of a tamper detection model provided by an embodiment of the embodiment;

[0031] Fig. 9 A schematic diagram of another structure of a tamper detection model provided in an embodiment;

[0032] Fig.10 A schematic diagram of a tampering detection process provided by an embodiment;

[0033] Fig.11 A schematic block diagram of a tamper detection device provided in an embodiment. DETAILED DESCRIPTION

[0034] The technical solutions of the embodiments of this specification will be described in detail below with reference to the accompanying drawings.

[0035] At present, there are some behaviors of tampering with cards (bank cards, identity documents, etc.) (for example, using paper and / or other facial images, handwriting pollution, etc. to physically cover the card information itself, or using image modification technology to modify the card information itself in the card image) in order to deceive the online card verification system of some institutions. Figure 1A A schematic diagram showing the result of the tampering method of modifying the card information by physically blocking the card with a piece of paper, wherein the area indicated by the arrow is the tampered area in the card; Figure 1B The schematic diagram shows the result of the tampering method that covers the card information by means of note pollution, where the area indicated by the arrow is the tampered area in the card; Figure 1C The figure shows the result corresponding to the tampering method of modifying the card information by modifying the obstruction through the image modification technology, wherein the information in the card pointed by the arrow on the left is the real information, and the information in the card pointed by the arrow on the right is the tampered information, that is, the tampered area in the card.

[0036] In order to ensure the security of the online card verification system of the aforementioned institution and improve the aforementioned institution's defense capability against attacks on tampered card images, the embodiments of this specification disclose a tampering detection method and device to achieve accurate detection of tampered areas in a card, so as to improve the aforementioned institution's defense capability against attacks on tampered card images.

[0037] in, Figure 2 A schematic diagram of an implementation scenario according to an embodiment disclosed in this specification is shown. In this implementation scenario, a card image to be detected is obtained, and the card image can be an image of any type of card, such as but not limited to: bank cards, credit cards, various types of identity documents, such as identity cards, driver's licenses, and passports.

[0038] Next, the location information of the target area in the card image is obtained, wherein the target area may be the area where the key object in the card is located, and the key object is an object with a high possibility of being tampered, such as a string and / or a face image specified in the card, wherein the specified string may include an identity identifier (such as ID), a name, a bank card number, etc. Then, the card image is input into a network for feature extraction (hereinafter referred to as a main branch network), and the card image is subjected to feature extraction through one or more first extraction layers in the network to obtain one or more first feature maps; then, based on the one or more first feature maps and the location information of the target area, a regional feature map corresponding to the target area is determined; through the first classification network, based on the regional feature map, a first detection result of whether the target area is a tampered area is obtained; at least based on the first detection result, a tampering detection result of whether the card image contains a tampered area is determined.

[0039] In the above process, for the target area in the card image (i.e., the area where the key object with a greater possibility of being tampered is located), based on its corresponding regional feature map, the first detection result of whether it is a tampered area is determined, which can achieve the focus on and detection of the local key area in the card image, and then based on the first detection result of the local key area in the image, determine whether the global card image contains the tampering detection result of the tampered area, thereby achieving the determination of the global tampering detection result of the card image, and improving the accuracy of the tampering detection result to a certain extent.

[0040] The tampering detection method provided in this specification is described in detail below in conjunction with specific embodiments.

[0041] Figure 3 The flowchart of a tamper detection method in one embodiment of the present specification is shown. The method is executed by an electronic device, and the electronic device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. In the tamper detection process, Figure 3 As shown, the method includes the following steps S310-S350:

[0042] In step S310, the position information of the target area in the card image to be detected is obtained. The target area may be the area where the key object is located in the card image. In some possible examples, the area where the key object is located includes the area where the specified string is located and / or the area where the face image is located. Accordingly, the key object may include the specified string and / or face image, which are generally objects in the card that are more likely to be tampered with. Exemplarily, in the case where the card image is an image corresponding to a bank card or a credit card, the key object may include but is not limited to at least one of the following: strings such as the bank card number and the name of the bank to which the bank card belongs. In the case where the card image is an image corresponding to an identity document, the key object may include but is not limited to at least one of the following: strings such as an identity identifier (such as an ID), a name, and an address, and objects such as a face image.

[0043] In some possible examples, the location information of the target area may be location information of the target area detected by the other device from the card image and obtained by the electronic device from the other device.

[0044] In some other possible examples, the electronic device may detect the target area in the card image based on a preset detection algorithm. In step S310, the following steps 11-14 may be included:

[0045] In step 11, the card image is detected based on a preset detection algorithm to obtain location information of each candidate target area in the card image, wherein the candidate target area includes an area suspected to be a key object area detected from the card image by the preset detection algorithm. Exemplarily, the area suspected to be a key object area may include any character string and / or face image area in the card image.

[0046] In some possible examples, the preset detection algorithm may be any algorithm in the related art that can detect character strings and / or face areas, and the algorithm may be, for example, a neural network model pre-trained based on sample images and their corresponding label data (including location information of character strings and / or face areas in the marked sample images). Exemplarily, the sample image may include an image obtained by photographing any card, or may include any image containing character strings and / or faces.

[0047] Exemplarily, the preset detection algorithm can be, but is not limited to, a model based on a DB (Differentiable Binarization) algorithm to achieve accurate detection of character strings in an image, such as a DBNet++ model; it can also be a special face detection model to achieve accurate detection of faces in an image; it can also be a detection algorithm based on Faster R-CNN (Faster Region-based Convolutional Network).

[0048] The electronic device can detect the card image based on a preset detection algorithm to obtain the location information of each candidate target area in the card image, that is, the area suspected to be the key object. In one case, the location information of each candidate target area can be displayed in the card image in the form of a marked box.

[0049] Then, in step 12, a specified region alignment algorithm is used to obtain a feature map of each candidate target region mapped to a first size based on the position information of each candidate target region. The first size is applicable to the second classification network and can be equal to the size of the input data of the second classification network.

[0050] In some possible examples, the electronic device may input the location information of each candidate target area and the card image into a specified area alignment algorithm, and utilize the specified area alignment algorithm to process each candidate target area in the card image based on the location information of each candidate target area to obtain a feature map corresponding to each candidate target area mapped to the first size.

[0051] In some other possible examples, the electronic device may input the position information of each candidate target area and the designated feature map corresponding to the card image into a designated area alignment algorithm, so as to utilize the designated area alignment algorithm to process the features corresponding to each candidate target area in the designated feature map corresponding to the card image based on the position information of each candidate target area, so as to obtain the feature map corresponding to each candidate target area mapped to the first size. Exemplarily, in the case where the preset detection algorithm is a neural network model for target detection, the designated feature map corresponding to the card image may be a feature map of a designated extraction layer in the neural network model, or a feature map obtained by fusing the feature maps of at least two designated extraction layers in the neural network model.

[0052] The specified region alignment algorithm includes, for example, a ROI (Region of Interest) Align algorithm.

[0053] In step 13, the classification result of whether each candidate target area is a target area is determined through a second classification network based on the feature map of the first size corresponding to each candidate target area. In this step, the electronic device can input the feature map of the first size corresponding to each candidate target area into the second classification network respectively, and process the feature map of the first size corresponding to each candidate target area through the second classification network to determine the classification result of whether each candidate target area is a target area. The second classification network can be any type of pre-trained network for classification (distinguishing whether the indicated area in the corresponding feature map is the target area, i.e., the area where the aforementioned key object is located), such as a multi-layer perceptron or a classifier based on other classification algorithms.

[0054] Then, in step 14, based on the aforementioned classification results, the location information of the target area is determined from the location information of each candidate target area. In this step, the aforementioned classification results can characterize which candidate target areas are the areas where the key objects are located, that is, the target areas. Accordingly, the electronic device, based on the aforementioned classification results, screens and determines the location information of the target area from the location information of each candidate target area, that is, determines the location information of the area where the key objects are located. In some examples, the target area can be one or more.

[0055] In step S320, feature extraction is performed on the card image through one or more first extraction layers to obtain one or more first feature maps. The one or more first extraction layers may belong to a network for feature extraction, such as the main branch network in the tampering detection model mentioned later, and the one or more first extraction layers are arranged in series to extract features from the input image. In this step, the electronic device inputs the card image into the one or more first extraction layers to extract features from the card image through the one or more first extraction layers to obtain one or more first feature maps. In the case of multiple first feature maps, they may be multi-scale feature maps, and there is a corresponding relationship between the first feature map and the first extraction layer.

[0056] After obtaining one or more first feature maps, in step S330, a regional feature map corresponding to the target area is determined based on the one or more first feature maps and the aforementioned position information.

[0057] In some possible implementations, the electronic device may determine the regional feature map corresponding to the target area based on a specified first feature map among the one or more first feature maps and the aforementioned position information of the target area. Specifically, the electronic device may convert the aforementioned position information of the target area from the size space of the card image to the size space of the specified feature map based on the size ratio between the specified first feature map and the card image, and then determine the regional feature map corresponding to the target area from the specified feature map based on the position information in the size space of the specified feature map.

[0058] In some other possible implementations, in order to obtain a more accurate detection result, the regional feature map corresponding to the target area can be jointly determined based on the multi-scale first feature map, so that the obtained regional feature map corresponding to the target area includes both global information and local detail information, thereby improving the accuracy of subsequent detection results. Specifically, the aforementioned first feature map is multiple; accordingly, in step S330, it can include 21-22:

[0059] In step 21, at least two designated first feature maps among the plurality of first feature maps are fused to obtain a fused feature map. In this step, at least two designated first feature maps are determined from the plurality of first feature maps, and then the sizes of the at least two designated first feature maps are unified, for example, according to the size of the feature map with the largest size among the at least two designated first feature maps, the feature map with a smaller size among the at least two designated first feature maps is upsampled to obtain at least two designated first feature maps with a unified size; for another example, according to the size of the feature map with the smallest size among the at least two designated first feature maps, the feature map with a smaller size among the at least two designated first feature maps is downsampled to obtain at least two designated first feature maps with a unified size.

[0060] After obtaining at least two designated first feature maps of uniform size, the at least two designated first feature maps of uniform size are fused (for example, pixel-by-pixel summing or splicing) to obtain a fused feature map.

[0061] Then, in step 22, based on the aforementioned position information of the target area and the size ratio between the fused feature map and the card image, the regional feature map corresponding to the target area is determined from the fused feature map.

[0062] In this step, the electronic device converts the location information of the target area from the size space of the card image to the size space of the fused feature map based on the size ratio between the fused feature map and the card image, and then determines the regional feature map corresponding to the target area from the fused feature map based on the location information converted to the size space of the fused feature map. This type of regional feature map contains richer information, which is more beneficial for subsequent tampering area detection.

[0063] After obtaining the regional feature map of the target area, in step S340, a first detection result of whether the target area is a tampered area is obtained based on the regional feature map through a first classification network. The first classification network can be any type of classification network. Exemplarily, the first classification network can be a multi-layer perceptron, which is used to determine whether the input feature map is a tampered area. In some possible examples, the electronic device can process the regional feature map through the first classification network to obtain a first detection result of whether the target area is a tampered area. In some examples, the first detection result can be a score value indicating whether the target area is a tampered area, wherein the larger the score value, the greater the possibility of indicating that the target area is a tampered area.

[0064] In some other possible examples, in order to better improve the accuracy of subsequent detection results, step S340 may include steps 31-32:

[0065] In step 31, the regional feature map is processed using a specified regional alignment algorithm to obtain a feature map corresponding to the regional feature map mapped to a second size. The second size is consistent with the size of the input data of the first classification network. The specified regional alignment algorithm is, for example, a ROI Align algorithm.

[0066] Then, in step 32, a first detection result of whether the target area is a tampered area is obtained based on the feature map of the second size corresponding to the regional feature map through the first classification network. In this step, the electronic device inputs the feature map of the second size corresponding to the regional feature map into the first classification network, and the first classification network processes the feature map of the second size corresponding to the regional feature map to obtain a first detection result of whether the target area is a tampered area.

[0067] In some possible examples, when there are multiple target areas, different target areas correspond to different first detection results.

[0068] Thereafter, in step S350, it is determined whether the card image contains a tampering detection result of the tampering area based at least on the first detection result.

[0069] In some possible examples, when any first detection result indicates that the corresponding target area is a tampered area, the tampering detection result may be determined as: the card image contains a tampered area.

[0070] In this embodiment, for the target area in the card image (i.e., the area where the key object with a high possibility of being tampered is located), based on its corresponding regional feature map, a first detection result of whether it is a tampered area is determined, which can achieve focused attention and detection on the local key area in the card image, and then based on the first detection result of the local key area in the image, determine whether the global card image contains a tampered area tampering detection result, thereby achieving determination of the global tampering detection result of the card image, and improving the accuracy of the tampering detection result to a certain extent.

[0071] In the above process, based on the regional feature maps corresponding to the target areas in the card image, i.e., the areas where the key objects are located, the feature maps obtained after being processed by the aforementioned specified regional alignment algorithm are used for tampering detection. For some tiny tampered areas in the card image (such as areas tampered with by tiny handwriting and areas tampered with by small foreign objects, etc.), the accuracy of the detection results is better improved.

[0072] Figure 4 A flowchart of a tamper detection method in another embodiment of the present specification is shown. The method is executed by an electronic device, and the electronic device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities.

[0073] exist Figure 4 In the tampering detection process shown, the detection results of the local areas that need to be focused on in the card image and the global segmentation detection results of the card image can be combined to determine whether the card image contains tampered areas, so as to better improve the accuracy of the tampering detection results and achieve global attention to the card image and attention to the local detail areas. Figure 4 In the tampering detection process shown, a tampering detection model needs to be used to achieve overall detection of the card image.

[0074] The following is an introduction to the training process of the tampering detection model. The tampering detection model includes: a main branch network, a reversible branch network, a first edge detection network and a first region detection network. The main branch network includes multiple first extraction layers (i.e., the aforementioned first extraction layer) connected in series, and the reversible branch network includes multiple second extraction layers connected in series. The first edge detection network pays more attention to the edge information of the tampered area in the image, and is a network for detecting the edge information of the tampered area in the image; the first region detection network pays more attention to the overall regional structure information of the tampered area in the image, and is a network for detecting the location of the tampered area in the image.

[0075] Before training the tampering detection model, a training data set S for training the tampering detection model can be constructed. The training data set S includes several sample images X and their label data Y. The sample image X can be a card image containing a tampered area (i.e., the sample image contains the tampered area), or a card image not containing a tampered area (i.e., the sample image does not contain the tampered area). The tampered area can include a card image that is marked with a paper strip (e.g., Figure 1A as shown) and / or other face images, handwriting contamination (such as Figure 1B The area that is obscured and tampered by physical obstruction tampering methods such as shown in the figure above; it can also include the area that is modified by image modification technology in the card (such as Figure 1C shown).

[0076] In some examples, the label data corresponding to each sample image may include a label segmentation map generated based on the tampered area in the sample image; and may also include a label edge map generated based on the tampered area in the sample image.

[0077] Among them, the label segmentation map can be a binary mask image. For a sample image containing a tampered area, this type of binary mask image can indicate the location of the tampered area in the corresponding sample image. Exemplarily, the pixel value of the pixel indicating the tampered area is a first value (for example, 1), and the pixel value of the pixel indicating the non-tampered area is a second value (for example, 0). For a sample image that does not contain a tampered area, this type of binary mask image can indicate that the corresponding sample image does not contain a tampered area, and the pixel values ​​of the pixels can all be the second value.

[0078] The label edge map may also be a binary image. For a sample image containing a tampered area, this type of binary image may indicate edge information of the tampered area in the corresponding sample image. For example, the pixel values ​​of the pixels indicating the edge of the tampered area are the first value, and the pixel values ​​of the pixels at the edge of other non-tampered areas are the second value. For a sample image that does not contain a tampered area, this type of binary image may indicate that the corresponding sample image does not contain a tampered area, and the pixel values ​​of the pixels may all be the second value.

[0079] In some other examples, the label data corresponding to each sample image may also include a label value indicating whether the sample image contains a tampered area. For example, a label value of a first value (e.g., 1) indicates that the sample image includes a tampered area, and a label value of a second value (e.g., 0) indicates that the sample image does not include a tampered area.

[0080] After obtaining the training data set S, the tampering detection model can be trained based on each sample image and its label data in the training data set S. It can be understood that the process of training the tampering detection model based on each sample image and its label data is similar. The following takes any first sample image Xj and its first label data Yj in the training data set S as an example to introduce the process of training the tampering detection model. The process of training the tampering detection model using other sample images and their label data can refer to the process of training the tampering detection model using the first sample image and its first label data. Specifically, the process of training the tampering detection model can include the following steps 41-46:

[0081] In step 41 , a first sample image and first label data thereof are obtained, wherein the first label data is used to indicate whether the first sample image contains a tampered area.

[0082] In some possible examples, the first label data may include a label segmentation map and a label edge map generated based on the first sample image. The label segmentation map may indicate whether the first sample image (i.e., the card) contains a tampered area. If the first sample image contains a tampered area, the label segmentation map in the first label data may indicate the location of the tampered area contained in the first sample image. The label edge map may indicate whether the first sample image contains a tampered area. If the first sample image contains a tampered area, the label edge map in the first label data may indicate the edge position of the tampered area contained in the first sample image. The aforementioned card may include, but is not limited to: bank cards, various types of identity documents, etc.

[0083] After the electronic device obtains the first sample image and its first label data, in step 42, it performs feature extraction on the first sample image based on multiple series-connected first extraction layers in the main branch network to obtain multiple second feature maps corresponding to multiple specified first extraction layers.

[0084] In this step, the electronic device can input the first sample image into the main branch network, and perform feature extraction on the first sample image based on the multiple first extraction layers connected in series in the main branch network to obtain multiple second feature maps corresponding to multiple specified first extraction layers. Among the multiple first extraction layers connected in series, the input of the first first extraction layer is the first sample image, and the input of each non-first first extraction layer is the output of the previous first extraction layer.

[0085] In some possible implementations, each first extraction layer of the main branch network may include but is not limited to operations for extracting features from an image, such as convolution operations, pooling operations, and batch normalization operations. In some examples, the main branch network may be an extraction network based on a Res2Net-50 structure to extract features from a first sample image, and obtain multiple second feature maps corresponding to multiple specified first extraction layers, and the sizes of the second feature maps corresponding to different specified first extraction layers may be different.

[0086] The aforementioned designated first extraction layer may at least include the first extraction layers corresponding to the third extraction layers in the reversible branch network mentioned later and the first extraction layers in the same layer as the third extraction layers. Figure 5 As shown, the main branch network may include 5 first extraction layers, including Figure 5 The extraction layers 11, 12, 13, 14 and 15 shown in FIG. 1 are shown in FIG. 1 . The designated first extraction layer may include various non-first-layer first extraction layers, for example, including Figure 5 The extraction layer 12, the extraction layer 13, the extraction layer 14 and the extraction layer 15 are shown.

[0087] In step 43, the first sample image is processed based on the reversible branch network to obtain multiple third feature maps corresponding to multiple specified second extraction layers, wherein the processing includes, through the third extraction layer among the multiple second extraction layers, performing feature extraction on the third feature map corresponding to its previous layer and the second feature map set corresponding to it to obtain its output feature map, and obtaining the corresponding third feature map based on the output feature map and the second feature map of the first extraction layer of the same layer, wherein the third extraction layer is a specified layer among the multiple second extraction layers starting from the second second extraction layer.

[0088] In this step, the electronic device inputs the first sample image into the reversible branch network, and processes the first sample image based on the multiple second extraction layers connected in series included in the reversible branch network, wherein the multiple second extraction layers connected in series include one or more third extraction layers, and the input of each third extraction layer includes the third feature map corresponding to the previous extraction layer and the second feature map set corresponding to the third extraction layer; the input of other second extraction layers except the third extraction layer in the multiple second extraction layers connected in series can be the first sample image or the third feature map corresponding to the previous extraction layer.

[0089] In some possible examples, for each third extraction layer in a plurality of second extraction layers connected in series (taking the third extraction layer P as an example for illustration), the third extraction layer P includes a fusion unit P1 and an extraction unit P2, and the aforementioned process of performing feature extraction on the third feature map a1 corresponding to the previous layer and the second feature map b1 set corresponding to the previous layer through the third extraction layer to obtain its output feature map may include: respectively adjusting the sizes of the third feature map a1 corresponding to the previous extraction layer and the second feature map b1 set corresponding to the third extraction layer (for example, upsampling and / or downsampling) to unify the sizes of the third feature map a1 corresponding to the previous extraction layer and the second feature map b1 set corresponding to the third extraction layer.

[0090] Next, the fusion unit P1 performs a first preset processing on the resized third feature map a2 corresponding to the previous extraction layer of the third extraction layer to obtain the corresponding processed third feature map a3, and performs a second preset processing on the resized second feature map b2 set corresponding to the third extraction layer to obtain the corresponding processed second feature map b3; the processed third feature map a3 and the processed second feature map b3 are subjected to a first fusion processing to obtain a fused feature map c of the third extraction layer; the extraction unit P2 performs feature extraction on the aforementioned fused feature map c to obtain the output feature map of the third extraction layer P.

[0091] For example, the first preset processing may include but is not limited to convolution operations and batch normalization operations, and the second preset processing may include but is not limited to linear processing, batch normalization operations, and difference processing, etc.; the aforementioned first fusion processing may include pixel-by-pixel addition of the processed third feature map and the processed second feature map, that is, adding the value of the pixel at the a-th row and b-th column position in the processed third feature map to the value of the pixel at the a-th row and b-th column position in the processed second feature map.

[0092] The aforementioned extraction unit P2 can be any extraction unit in the related art that can realize feature extraction, such as a feature extraction unit based on a residual network, or a feature extraction unit based on a ConvNeXt network, etc.

[0093] Next, after obtaining the output feature map through the third extraction layer P, a preset reversible processing is performed on the second feature map of the first extraction layer on the same layer as the third extraction layer P to obtain the feature map after the reversible processing, and a second fusion processing is performed on the output feature map of the third extraction layer P and the feature map after the reversible processing to obtain the third feature map corresponding to the third extraction layer P.

[0094] The preset reversible processing may refer to an operation in which the corresponding output can obtain the corresponding input through the inverse operation of the preset reversible processing. The reversible processing may include, but is not limited to, multiplying the second feature map of the first extraction layer at the same layer as the third extraction layer P by a preset coefficient; adding the second feature map of the first extraction layer at the same layer as the third extraction layer P to specified data, etc. The second fusion processing may be pixel-by-pixel addition of the output feature map of the third extraction layer P and the feature map after the reversible processing.

[0095] In some examples, the structure of the aforementioned other second extraction layer may be similar to the structure of the aforementioned third extraction layer, wherein the second feature map corresponding to it (the aforementioned other second extraction layer) may be empty. In some other examples, the structure of the aforementioned other second extraction layer may be similar to the structure of the first extraction layer.

[0096] In some possible examples, the number of the multiple second extraction layers of the reversible branch network may be the same as or different from the number of the multiple first extraction layers of the main branch network; when different, the number of the multiple second extraction layers of the reversible branch network may be less than the number of the multiple first extraction layers of the main branch network. Figure 5 As shown, the number of the second extraction layers of the reversible branch network can be the same as the number of the first extraction layers of the main branch network, both of which are 5 layers. Figure 5 As shown, the reversible branch network includes extraction layer 21, extraction layer 22, extraction layer 23, extraction layer 24 and extraction layer 25.

[0097] In some implementations, the third extraction layer may be set as required, for example, it may include one or more third extraction layers, and the third extraction layer may be an extraction layer with a higher number of layers in the reversible branch network, or an extraction layer with a lower number of layers in the reversible branch network. Figure 5 As shown, the third extraction layer can be the third extraction layer to the fifth extraction layer in the reversible branch network, namely, extraction layer 23, extraction layer 24 and extraction layer 25.

[0098] In some possible examples, the third extraction layer is the i-th extraction layer in the reversible branch network; the second feature map corresponding to the third extraction layer can be the second feature map of the i+n-th first extraction layer in the main branch network, where n is a positive integer. The specific value of n can be set according to requirements and the number of first extraction layers in the main branch network.

[0099] For example, n can be set to 1 or 2, etc. Figure 5 As shown, assuming that n is set to 1, for the extraction layer 23 in the reversible branch network, the second feature map corresponding to the third extraction layer is set to be the second feature map of the first extraction layer of the 3+nth layer (n is 1) in the main branch network, that is, the fourth first extraction layer in the main branch network (that is, Figure 5 The second characteristic map of the extraction layer 14 shown in the figure; for the extraction layer 24 in the reversible branch network, the second characteristic map corresponding to the third extraction layer is set to the fifth first extraction layer in the main branch network (that is, as shown in the figure); Figure 5 The second characteristic map of the extraction layer 15 shown; for the extraction layer 25 in the reversible branch network, the second characteristic map corresponding to the third extraction layer is empty.

[0100] The aforementioned multiple designated second extraction layers can be set according to requirements, and the multiple designated second extraction layers include, for example: the non-first layer second extraction layers in the multiple second extraction layers. Figure 5 The extraction layer 22, the extraction layer 23, the extraction layer 24 and the extraction layer 25 are shown.

[0101] After the electronic device obtains a plurality of third feature maps corresponding to a plurality of specified second extraction layers, in step 44 , a second tampering edge map is obtained based on the plurality of third feature maps through a first edge detection network.

[0102] In some possible examples, considering the third feature maps corresponding to different designated second extraction layers, the shallower the designated second extraction layer is in the reversible branch network, the more local edge detail information in the image contained in the corresponding third feature map, which can provide more edge information of the tampered area; the deeper the designated second extraction layer is in the reversible branch network, the more global information in the image (information related to the location of each tampered area) contained in the corresponding third feature map, that is, the supervision of the edge positioning of the tampered area can be better achieved. In order to enable the first edge detection network to predict the edge information of the tampered area with higher accuracy, the electronic device can input the multiple third feature maps corresponding to the multiple designated second extraction layers into the first edge detection network, and obtain the second tampered edge map based on the multiple third feature maps through the first edge detection network, so as to better mine the edge information of the tampered area.

[0103] In some possible examples, based on the above situation and taking into account the model training efficiency, the aforementioned multiple third feature maps may include: a feature map corresponding to the target second extraction layer of the reversible branch network and a feature map corresponding to the last second extraction layer, and the number of layers of the target second extraction layer in the reversible branch network is not higher than the preset number of layers; accordingly, in step 44, the following steps are included: the feature map corresponding to the target second extraction layer and the feature map corresponding to the last second extraction layer are input into the first edge detection network to determine the second tampering edge map.

[0104] Among them, the number of layers of the target second extraction layer in the reversible branch network is not higher than the preset number of layers, and its corresponding feature map contains more local edge detail information in the image, and the feature map corresponding to the last second extraction layer contains more global information in the image. The electronic device inputs the feature map corresponding to the target second extraction layer and the feature map corresponding to the last second extraction layer into the first edge detection network, so that the first edge detection network can mine the edge semantics related to the tampered area with better accuracy from the feature map corresponding to the target second extraction layer containing more local edge detail information in the image (also called low-level features) and the feature map corresponding to the last second extraction layer containing more global information under the supervision of the tampered area boundary (also called high-level features), thereby obtaining a second tampered edge map with higher accuracy.

[0105] In one implementation, the aforementioned target second extraction layer may be the second extraction layer in the reversible branch network, such as Figure 5 and Figure 6 As shown, the feature map corresponding to the target second extraction layer is feature map f22, and the feature map corresponding to the last second extraction layer is feature map f25.

[0106] Exemplarily, the first edge detection network may be any network model in the related art that can realize edge detection. In some possible examples, the structure of the first edge detection network may be as follows: Figure 6 As shown, it may include a first convolutional layer and a second convolutional layer arranged in parallel, wherein the input of the first convolutional layer is the feature map corresponding to the aforementioned target second extraction layer (e.g., feature map f12), and the input of the second convolutional layer is the feature map corresponding to the aforementioned last second extraction layer (e.g., feature map f15). The second convolutional layer is followed by an upsampling layer "U", which is then connected to a concatenation layer "C"; the first convolutional layer is connected to the concatenation layer; and the concatenation layer is followed by a plurality of third convolutional layers and an activation layer. The sizes of the convolution kernels of the first convolutional layer and the second convolutional layer may both be 1*1.

[0107] Exemplarily, the third convolutional layers connected to the concatenated layer may be one or more. In some possible examples, for example Figure 6As shown, the plurality of third convolutional layers may include a plurality of serially connected convolutional layers, specifically including: a third convolutional layer 1 with two serially connected convolution kernels of size 3*3 and a third convolutional layer 2 with a convolution kernel size of 1*1. The aforementioned splicing layer is provided with a splicing function (for example, a concat function) to splice its two inputs. The activation layer is provided with an activation function (for example, a Sigmoid function). Through the above structure of the first edge detection network, based on the local edge detail information contained in the feature map corresponding to the target second extraction layer and the global information contained in the feature map corresponding to the last second extraction layer, a second tampered edge map with higher accuracy can be determined.

[0108] Then, in step 45, a plurality of second segmentation maps are obtained based on the plurality of third feature maps and the second tampering edge map through the first region detection network.

[0109] In this step, the electronic device can integrate the second tampered edge map with each second feature map, i.e., each level feature, through the first area detection network, and guide each level feature to learn the edge information therein through the second tampered edge map to enhance the edge information in each level feature; and then obtain multiple second segmented images based on each level feature after edge information enhancement through the first area detection network.

[0110] Some possible examples include Figure 5 As shown, the first area detection network may include: a plurality of first detection sub-networks connected in series (including Figure 5 , "CAM1", "CAM2" and "CAM3" as shown), and the first edge enhancement sub-networks corresponding to the specified second extraction layers (including Figure 5 As shown, "EFM1" corresponding to the extraction layer 22 is input as the second feature map f22 corresponding to the extraction layer 22; "EFM2" corresponding to the extraction layer 23 is input as the second feature map f23 corresponding to the extraction layer 23; "EFM3" corresponding to the extraction layer 24 is input as the second feature map f24 corresponding to the extraction layer 24; "EFM4" corresponding to the extraction layer 25 is input as the second feature map f25 corresponding to the extraction layer 25); accordingly, step 45 may include the following steps 451-452:

[0111] In step 451, the third feature map and the second tampered edge map corresponding to each first edge enhancement sub-network are processed respectively to obtain the edge enhancement feature map of each first edge enhancement sub-network. It can be understood that the second extraction layer is specified to correspond to the first edge enhancement sub-network, the second extraction layer is specified to correspond to the third feature map, and accordingly, the first edge enhancement sub-network corresponds to the third feature map.

[0112] In this step, the electronic device combines the third feature maps corresponding to each specified second extraction layer with the second tampered edge map respectively to obtain multiple image groups, each image group including a third feature map corresponding to a specified second extraction layer and a second tampered edge map; each image group is input into each first edge enhancement sub-network corresponding to each specified second extraction layer, so that each first edge enhancement sub-network processes its corresponding third feature map and second tampered edge map respectively, and integrates the edge information of the second tampered edge map with the features in the corresponding third feature map, that is, injects the edge information of the second tampered edge map into the representation learning of the corresponding third feature map to enhance the edge feature representation in the corresponding third feature map, and obtains the edge enhanced feature map of each first edge enhancement sub-network.

[0113] In some possible examples, the first edge enhancement sub-network may be any neural network that can realize feature fusion integration. Exemplarily, in order to achieve good integration and obtain edge enhancement feature maps of each first edge enhancement sub-network with better edge enhancement effect, the first edge enhancement sub-network may introduce a local attention mechanism, and utilize the introduced local attention mechanism to perform feature interaction and integration between the corresponding third feature map and the second tampered edge map. Figure 7 As shown, it is a structural schematic diagram of an arbitrary first edge enhancement sub-network k (k is a positive integer), which includes a downsampling layer, a first fusion layer, a second fusion layer, a fifth convolutional layer, a processing layer based on a local attention mechanism, a third fusion layer and a sixth convolutional layer connected in sequence.

[0114] like Figure 7 As shown, the input of the downsampling layer is the second tampered edge map; the first fusion layer is set to pixel-by-pixel multiplication processing, and its input is the corresponding third feature map and the second tampered edge map after downsampling by the downsampling layer; the second fusion layer is set to pixel-by-pixel addition processing, and its input is the corresponding third feature map and the output of the first fusion layer; the input of the fifth convolutional layer is the output of the second fusion layer, and the size of its convolution kernel is set to 3*3, for example; the processing layer based on the local attention mechanism may include a pooling sublayer (for example, a GAP (Global Average Pooling) layer), a convolution sublayer and an activation sublayer (in which, for example, a Sigmoid activation function is set); the third fusion layer is set to pixel-by-pixel multiplication, and its input is the output of the fifth convolutional layer and the output of the processing layer based on the local attention mechanism; the input of the sixth convolutional layer is the output of the third fusion layer.

[0115] The electronic device obtains the edge enhancement feature map of each first edge enhancement subnetwork, and then, in step 452, processes the edge enhancement feature map and the corresponding first input data of the corresponding first edge enhancement subnetwork through any target detection subnetwork among the multiple first detection subnetworks to obtain a second segmentation map corresponding to the target detection subnetwork, wherein, when the target detection subnetwork is the first first detection subnetwork, its corresponding first input data is the edge enhancement feature map of the first edge enhancement subnetwork specified among the multiple first edge enhancement subnetworks; when the target detection subnetwork is not the first first detection subnetwork, its corresponding first input data is the output of the previous first detection subnetwork.

[0116] In this step, the electronic device can use each first detection subnetwork as a target detection subnetwork in turn according to the serial connection relationship of multiple first detection subnetworks, input the edge enhancement feature map and the first input data of the first edge enhancement subnetwork corresponding to the target detection subnetwork into the target detection subnetwork, and process the edge enhancement feature map and the corresponding first input data of the first edge enhancement subnetwork corresponding to the target detection subnetwork through the target detection subnetwork to obtain a second segmentation map corresponding to the target detection subnetwork.

[0117] Wherein, when the target detection subnetwork is the first first detection subnetwork, its corresponding first input data is the edge enhancement feature map of the first edge enhancement subnetwork specified in the multiple first edge enhancement subnetworks. Exemplarily, when the target detection subnetwork is the first first detection subnetwork, the aforementioned specified first edge enhancement subnetwork can be the first edge enhancement subnetwork corresponding to the second feature map corresponding to the last second extraction layer. When the target detection subnetwork is not the first first detection subnetwork, its corresponding first input data is the output of the previous first detection subnetwork.

[0118] By setting up multiple first detection sub-networks connected in series to mine features in each level (context), that is, using the detection result of the previous first detection sub-network, the current first detection sub-network is guided to obtain the detection result from its corresponding edge enhancement feature map to enhance the detection of the tampered area. The first detection sub-network can be any neural network in the relevant technology that can realize the detection of the tampered area.

[0119] Thereafter, in step 46 , a tampering detection model is trained based on each second segmentation map and the first label data.

[0120] The first label data may include a label segmentation map generated based on the first sample image. At this time, the electronic device may construct a total loss based on the difference between each second segmentation map and the label segmentation map, and then adjust the model parameters of the tampering detection model to train the tampering detection model with the goal of minimizing the total loss, that is, minimizing the difference between each second segmentation map and the label segmentation map.

[0121] In some other examples, the first label data includes a label segmentation map and a label edge map generated based on the first sample image. Accordingly, in step 46, it may include: training a tampering detection model based on the second tampering edge map, each second segmentation map, and the label tampering edge map and label segmentation map in the first label data. In this example, the electronic device may jointly determine the total loss based on the difference between the second tampering edge map and the label edge map, and the difference between each second segmentation map and the label segmentation map, and then adjust the model parameters of the tampering detection model to train the tampering detection model with the goal of minimizing the total loss, that is, minimizing the difference between the second tampering edge map and the label edge map, and the difference between each second segmentation map and the label segmentation map.

[0122] In some possible examples, the aforementioned first label data includes a label tampering edge map and a label segmentation map generated based on the first sample image; accordingly, step 46 includes steps 461-464:

[0123] In step 461, a first loss is constructed based on the difference between the second tampered edge map and the label tampered edge map. In some possible examples, the electronic device can construct a first loss based on the difference between the second tampered edge map and the label tampered edge map based on a preset first loss function for measuring the similarity between two objects, wherein the first loss is negatively correlated with the similarity between the second tampered edge map and the label tampered edge map, and accordingly, the higher the similarity between the second tampered edge map and the label tampered edge map, the smaller the first loss. For example, the first loss function is, for example, a dice loss function.

[0124] In step 462, based on the difference between the label segmentation map and each second segmentation map, a second loss is constructed. In this step, the electronic device may use a weighted binary cross entropy loss function and / or a weighted IOU (Intersection over Union) loss function to construct sub-losses corresponding to each second segmentation map based on the difference between the label segmentation map and each second segmentation map, and then determine the second loss based on the sub-losses corresponding to each second segmentation map. Exemplarily, the sum of the sub-losses corresponding to each second segmentation map is determined as the second loss; or the average value between the sub-losses corresponding to each second segmentation map is determined as the second loss.

[0125] In step 463, a total loss is constructed, wherein the total loss is positively correlated with the first loss and the second loss. Then, in step 464, the tampering detection model is trained with the goal of minimizing the total loss.

[0126] Specifically, the electronic device may determine the total loss as the product of the first loss and the corresponding weight value, and the sum of the product of the second loss and the corresponding weight value; then, with the goal of minimizing the total loss, that is, minimizing the difference between the second tampered edge map and the label tampered edge map, and minimizing the difference between the label segmentation map and each second segmentation map, the model parameters of the main branch network, the reversible branch network, the first edge detection network and the first area detection network in the tampering detection model are adjusted to train the tampering detection model.

[0127] The above steps 41-46 are a model iteration training process for the tamper detection model. In order to train a better tamper detection model, the above process can be iterated multiple times. That is, after step 46, based on the updated model parameters of the tamper detection model, return to step 41.

[0128] The stopping conditions of the above-mentioned model iterative training process may include: the number of iterative training times reaches a preset threshold, or the iterative training duration reaches a preset duration, or the total loss is less than a set loss threshold, etc.

[0129] In the above process, setting up a reversible branch network can alleviate the information bottleneck problem caused by the excessive depth of the neural network based on deep learning. That is to say, the depth of the main branch network does not need to be too deep (that is, there is no need for too many first extraction layers in the main branch network), and it can also be combined with the assistance of the reversible branch network to generate reliable gradient information. Based on this type of gradient information, the main branch network can learn features that are more useful for the tampering area detection task, and solve the information bottleneck problem caused by the excessive depth of the neural network based on deep learning.

[0130] Moreover, for the tampering detection model, the feature maps corresponding to the second extraction layers of different layers in the reversible branch network can be used to perform different detection tasks, for example, tampering regions of different sizes can be detected respectively, wherein the second extraction layers with smaller layers can be guided to learn the features of smaller tampering regions in the image, and the second extraction layers with larger layers can be guided to learn the features of larger tampering regions in the image. Furthermore, when constructing the total loss, the second segmentation maps corresponding to the multiple second extraction layers output by the first region detection network are combined, so that the total loss aggregates the information of the tampering regions of the multiple layers of the second extraction layers in the reversible branch network, and then based on the gradient information obtained by the total loss, the gradient information of the tampering regions corresponding to the multiple layers of the second extraction layers in the reversible branch network is aggregated, and then reversely transmitted to the main branch network, so that the main branch network can learn the features of tampering regions of various sizes, and can retain more local detail information, so that the tampering detection model can realize tampering regions of different sizes, and improve the accuracy of the tampering region detection results of the tampering detection model.

[0131] In some other possible examples, the first label data may further include a label value for indicating whether the first sample image contains a tampered area; Figure 8 As shown, the tampering detection model includes a main branch network, a reversible branch network, a first edge detection network, a first region detection network and a tampering region classification network, wherein the main branch network includes multiple first extraction layers connected in series, and the reversible branch network includes multiple second extraction layers connected in series.

[0132] In some possible examples, Figure 8 Based on the structure shown, in the training process of the tampering detection model, after the electronic device obtains each second segmentation map through the aforementioned steps 41-45, in step 46, the following steps 51-52 may be included: In step 51, using the tampering region classification network, based on the target segmentation image in each second segmentation image, a prediction value for indicating whether the first sample image contains a tampered region is determined. The target segmentation image may be all the second segmentation images, or the segmentation map output by the last first detection subnetwork in each second segmentation image.

[0133] Exemplarily, the tampered region classification network may include at least a classifier (e.g., a multilayer perceptron or other classifier or a fully connected layer) to determine, through the classifier, a prediction value indicating whether the first sample image contains a tampered region based on the target segmented image.

[0134] Then, in step 52, the tampering detection model is trained based on the difference between the label value and the predicted value. In this step, a total loss is constructed based on the difference between the label value and the predicted value, and the model parameters of the tampering detection model are adjusted to train the tampering detection model with the goal of minimizing the total loss, that is, minimizing the difference between the label value and the predicted value.

[0135] In some possible examples, Figure 8 Based on the structure shown, in the training process of the tampering detection model, after the electronic device obtains each second segmentation image and the second tampering edge image through the aforementioned steps 41-46, in step 46, the following steps 61-62 may be included: In step 61, a tampering region classification network is used to determine a prediction value indicating whether the first sample image contains a tampered region based on the target segmentation image and the second tampering edge map in each second segmentation image. Then, in step 62, the tampering detection model is trained based on the difference between the label value and the prediction value.

[0136] In one implementation, the tampered region classification network may include two feature extraction layers a arranged in parallel, a feature fusion layer connected to the two feature extraction layers a, and a classifier connected to the feature fusion layer. Accordingly, the electronic device may input the target segmentation image into a feature extraction layer a1; and input the second tampered edge map into another feature extraction layer a2; then the outputs of the two feature extraction layers are input into the feature fusion layer to obtain the fused feature map; and the fused feature map is processed by the classifier to obtain the predicted value. Then, based on the difference between the label value and the predicted value, the tampering detection model is trained. The feature extraction layer may be implemented based on any network capable of feature extraction in the relevant technology. In some implementations, the feature extraction layer may include a GeM pooling layer (Generalized Mean pooling layer) to extract more features that are beneficial to the tampered region detection task from the target segmentation image and the second tampered edge map, thereby improving the accuracy of the predicted value. The feature fusion layer may achieve the fusion of the two inputs by adding the two inputs pixel by pixel, or multiplying or splicing the two inputs pixel by pixel.

[0137] In another implementation, the tampered region classification network may include a first classification subnetwork and a second classification subnetwork, the electronic device may process the target segmentation image through the first classification subnetwork to obtain a first value, process the second tampered edge map through the second classification subnetwork to obtain a second value, perform weighted summation on the first value and the second value to obtain a predicted value, and then train the tampering detection model based on the difference between the label value and the predicted value. Each classification subnetwork may include at least a classifier; a feature extraction layer may also be included before the classifier.

[0138] In another implementation, the tampering region classification network may also include a third classification subnetwork, and the electronic device may also process the feature map of the last second extraction layer in the plurality of third feature maps through the third classification subnetwork to obtain a third value; then perform weighted summation on the first value, the second value, and the third value to obtain a predicted value; and then train the tampering detection model based on the difference between the label value and the predicted value. This is to achieve a joint decision-making mechanism that comprehensively considers multiple information channels, so as to better improve the tampering detection accuracy of the tampering detection model. The structure of the third classification subnetwork can refer to the structures of the first classification subnetwork and the second classification subnetwork.

[0139] In another implementation, after training the main branch network, reversible branch network, first edge detection network and first area detection network in the tampering detection model, the classification network can be trained based on the aforementioned sample images and their label data. When training the tampering area classification network, the model parameters of the main branch network, reversible branch network, first edge detection network and first area detection network can be fixed.

[0140] In some examples, the present specification provides a method for training a tamper detection model in another embodiment. Fig. 9 As shown, the tamper detection model includes a main branch network, a reversible branch network, a first edge detection network, a first area detection network, a second edge detection network, and a second area detection network. The main branch network includes a plurality of first extraction layers connected in series, and the reversible branch network includes a plurality of second extraction layers connected in series. During the training process of the tamper detection model, the method includes the following steps 71-78:

[0141] In step 71, a first sample image and first label data thereof are obtained, wherein the first sample image is a card image containing a tampered area.

[0142] In step 72, feature extraction is performed on the first sample image based on the main branch network to obtain multiple second feature maps corresponding to multiple specified first extraction layers.

[0143] In step 73, the first sample image is processed based on the reversible branch network to obtain multiple third feature maps corresponding to multiple specified second extraction layers, wherein the processing includes, through the third extraction layer among the multiple second extraction layers, performing feature extraction on the third feature map corresponding to its previous layer and the second feature map set corresponding to it to obtain its output feature map, and obtaining the corresponding third feature map based on the output feature map and the second feature map of the first extraction layer of the same layer.

[0144] In step 74, a second tampering edge map is obtained based on the plurality of third feature maps through the first edge detection network.

[0145] Among them, the implementation principle of steps 71-74 is similar to the implementation principle of steps 41-44 shown above. The implementation process can refer to the implementation process of steps 41-44, and will not be repeated here.

[0146] In step 75, a third tampering edge map is obtained based on the plurality of second feature maps through a second edge detection network. Fig. 9 As shown, the structure of the second edge detection network is similar to that of the first edge detection network, and its specific structure can refer to the specific structure of the first edge detection network. The implementation principle of step 75 is similar to that of the aforementioned step 74, and its implementation process can refer to the implementation process of step 74, which will not be repeated here.

[0147] In step 76, a plurality of second segmentation maps are obtained based on the plurality of third feature maps and the second tampering edge map through the first region detection network, wherein the second segmentation map includes the predicted tampering region in the first sample image. The implementation principle of step 76 is similar to the implementation principle of step 45 shown above, and its implementation process can refer to the implementation process of step 45, which will not be described in detail here.

[0148] In step 77, a plurality of third segmentation maps are obtained based on the third tampering edge map and the plurality of second feature maps through the second region detection network. The structure of the second region detection network is similar to that of the first region detection network, such as Fig. 9 As shown, the second edge detection network includes a plurality of second detection sub-networks connected in series (including Fig. 9 , "CAM4", "CAM5" and "CAM6" as shown), and the second edge enhancement sub-networks corresponding to the specified first extraction layers (including Fig. 9 As shown, "EFM5" corresponding to extraction layer 12 is input as the second feature map f12 corresponding to extraction layer 12; "EFM6" corresponding to extraction layer 13 is input as the second feature map f13 corresponding to extraction layer 13; "EFM7" corresponding to extraction layer 14 is input as the second feature map f14 corresponding to extraction layer 14; "EFM8" corresponding to extraction layer 15 is input as the second feature map f15 corresponding to extraction layer 15), wherein the structures of the second detection subnetwork and the second edge enhancement subnetwork are similar to the structures of the first detection subnetwork and the first edge enhancement subnetwork in the first region detection network, respectively, and their specific structures can refer to the specific structures of the first detection subnetwork and the first edge enhancement subnetwork, respectively. The implementation principle of step 77 is similar to that of the aforementioned step 76, and its implementation process can refer to the implementation process of step 76, which will not be repeated here.

[0149] In step 78 , a tampering detection model is trained based on each second segmentation map, each third segmentation map and the first label data.

[0150] Exemplarily, the first label data includes a label segmentation map generated based on the first sample image. Accordingly, the electronic device can construct sub-losses corresponding to each segmentation map (including the second segmentation map and the third segmentation map) based on the difference between the label segmentation map and each second segmentation map and each third segmentation map; then, the sub-losses are combined to determine the total loss, and the total loss is positively correlated with each sub-loss; then, the tampering detection model is trained with the goal of minimizing the total loss, that is, minimizing each sub-loss.

[0151] In another exemplary embodiment, the first label data includes a label edge map and a label segmentation map generated based on the first sample image. Accordingly, in step 78, the electronic device can construct a first loss based on the difference between the second tampered edge map and the label tampered edge map; construct a second loss based on the difference between the label segmentation map and each second segmentation map; construct a third loss based on the difference between the third tampered edge map and the label tampered edge map; construct a fourth loss based on the difference between the label segmentation map and each third segmentation map; combine the first loss, the second loss, the third loss and the fourth loss to construct a total loss, wherein the total loss is positively correlated with the first loss, the second loss, the third loss and the fourth loss respectively; then, with the goal of minimizing the total loss, adjust the model parameters of the main branch network, the reversible branch network, the first edge detection network, the first region detection network, the second edge detection network and the second region detection network in the tampering detection model to train the tampering detection model.

[0152] The specific process of determining the third loss and the fourth loss may refer to the specific process of determining the first loss and the second loss mentioned above, which will not be elaborated here.

[0153] In the above process, the feature maps corresponding to the second extraction layers of different layers in the reversible branch network of the tampering detection model can be used to perform different detection tasks, and the feature maps corresponding to the first extraction layers of different layers in the main branch network can be used to perform different detection tasks, for example, tampering areas of different sizes can be detected respectively, wherein the second extraction layer (first extraction layer) with a smaller number of layers can be guided to learn the features of smaller tampering areas in the image, and the second extraction layer (first extraction layer) with a larger number of layers can be guided to learn the features of larger tampering areas in the image. Subsequently, when constructing the total loss, the second segmentation maps corresponding to the multiple second extraction layers output by the first region detection network and the third segmentation maps corresponding to the multiple first extraction layers output by the second region detection network are combined, so that the total loss aggregates the information of the tampered areas of the multiple layers of second extraction layers in the reversible branch network and the information of the tampered areas of the multiple layers of first extraction layers in the main branch network, and then based on the gradient information obtained by the total loss, the gradient information of the tampered areas corresponding to the multiple layers of second extraction layers in the reversible branch network and the gradient information of the tampered areas of the multiple layers of first extraction layers in the main branch network are aggregated, and then transmitted back to the main branch network, so that the main branch network does not need to set more first extraction layers, and can still better learn the characteristics of tampered areas of various sizes, and can retain more local detail information, so that the tampering detection model can realize tampering areas of different sizes, better improve the accuracy of the tampering area detection results of the tampering detection model, and solve the information bottleneck problem caused by the excessive depth of the neural network based on deep learning.

[0154] In some possible examples, the tampering detection model may also include the aforementioned first classification network, and accordingly, at least the aforementioned parts in the tampering detection model (such as the aforementioned main branch network, reversible branch network, first edge detection network, first area detection network, etc.) and the first classification network may be jointly trained. At this time, in the training data used for jointly training the tampering detection model, the label data corresponding to each sample image may also include the location information of the area where each sample key object in the sample image is located (for example, in the form of a marked box, referred to as label location information) and label data indicating whether the area where each sample key object is located is a tampered area. When constructing the loss for adjusting the tampering detection model, in addition to constructing based on the aforementioned losses for training the tampering detection model, it is also necessary to construct for each sample image based on the difference between the label data indicating whether the area where each sample key object is located in the sample image is a tampered area and the prediction result data output by the first classification network for the sample image indicating whether the area where each sample key object is located is a tampered area.

[0155] Specifically, the tampering detection model may further include a first classification network; the aforementioned first label data may include a label map generated based on the first sample image (the label map may include the aforementioned label segmentation map, and may also include the aforementioned label edge map), as well as label position information of the area where each sample key object is located in the first sample image, and label data indicating whether the area where each sample key object is located is a tampered area;

[0156] Based on the process of training the tampering detection model shown in the aforementioned steps 41-45, the process of training the tampering detection model may further include the following steps 81-82:

[0157] In step 81, based on the aforementioned multiple second feature maps and label position information, the sample area feature map corresponding to the area where each sample key object is located is determined. The implementation principle of step 81 is similar to the implementation principle of the aforementioned step S330, and its implementation process can refer to the implementation process of the aforementioned step S330, which will not be repeated here.

[0158] Afterwards, in step 82, the prediction result data of whether the area where each sample key object is located is a tampered area is obtained through the first classification network based on the feature map of each sample area. The implementation principle of step 82 is similar to the implementation principle of the aforementioned step S340, and its implementation process can refer to the implementation process of the aforementioned step S340, which will not be repeated here.

[0159] In step 46, a tampering detection model is trained based on each second segmentation map and label map, as well as the prediction result data and label data indicating whether the area where each sample key object is located is a tampered area.

[0160] In this step, the electronic device can construct a sub-loss corresponding to each second segmentation map based on the difference between each second segmentation map and the label map, wherein the process of constructing the sub-loss corresponding to each second segmentation map can refer to the aforementioned process of constructing the sub-loss corresponding to each second segmentation map based on the difference between each second segmentation map and the label segmentation map, and will not be repeated here.

[0161] The electronic device may adopt a cross entropy loss function to construct a regional loss based on the difference between the prediction result data and the label data indicating whether the area where each sample key object is located is a tampered area. Then, based on the regional loss and the sub-loss corresponding to each second segmentation map, the total loss is jointly determined, and the total loss is positively correlated with the sub-loss and regional loss corresponding to each second segmentation map. Then, with the goal of minimizing the total loss, the model parameters of each part of the tampering detection model are adjusted.

[0162] It is understandable that the main branch network, the reversible branch network, the first edge detection network, the first region detection network, the second edge detection network, the second region detection network, the tampering region classification network and the first classification network in the tampering detection model can also be trained together to better improve the accuracy of the detection results of the tampering detection model. The corresponding training process can refer to the aforementioned training process, which will not be repeated here.

[0163] After the tampering detection model is trained in the above manner, the main branch network, the first edge detection network, the first region detection network and the first classification network in the tampering detection model can be combined with the location information of the target region in the card image to detect whether the card image contains a tampered region. Figure 4 As shown, the method includes the following steps S410-S450:

[0164] In step S410, the position information of the target area in the card image to be detected is obtained, wherein the target area is the area where the key object in the card image is located.

[0165] In step S420, feature extraction is performed on the card image through one or more first extraction layers to obtain one or more first feature maps. The one or more first extraction layers are first extraction layers connected in series in the aforementioned main branch network. The one or more first feature maps are multiple first feature maps obtained by multiple specified first extraction layers in the main branch network based on the card image.

[0166] In step S430, based on the one or more first feature maps and the location information, a regional feature map corresponding to the target area is determined. Exemplarily, the regional feature map corresponding to the target area can be determined based on the first feature map specified in the one or more first feature maps and the location information of the target area. Fig.10 As shown, the specified first feature map may be the first feature map fa3 output by the extraction layer 13 in the middle position in the main branch network. In another exemplary embodiment, the regional feature map corresponding to the target area may be determined based on at least two specified first feature maps in the one or more first feature maps and the location information of the target area. For example, the at least two specified first feature maps may include Fig.10 The first feature map fa3 output by the extraction layer 13 and the first feature map output by the extraction layer 14 are shown.

[0167] In step S440, a first detection result of whether the target area is a tampered area is obtained based on the regional feature map through the first classification network. Fig.10The “region alignment algorithm” shown in the figure) processes the regional feature map to obtain a feature map mapped to the second size corresponding to the regional feature map, and then, through the first classification network, obtains a first detection result of whether the target area is a tampered area based on the feature map of the second size corresponding to the regional feature map.

[0168] Among them, the implementation principle of steps S410-S440 is the same as Figure 3 The implementation principles of steps S310-S340 are similar, and the implementation process can refer to the implementation process of steps S310-S340, which will not be repeated here.

[0169] In step S450, a first tampering edge map is obtained based on one or more first feature maps through a first edge detection network. The implementation principle of step S450 is similar to that of the aforementioned step 44. The implementation process thereof can refer to the implementation process of the aforementioned step 44, and will not be described in detail here. Fig.10 As shown, the first edge detection network can be used based on the feature maps corresponding to the target first extraction layer in multiple first feature maps (such as Fig.10 The feature map corresponding to the second first extraction layer as shown in FIG2) and the feature map corresponding to the last first extraction layer as shown in FIG3 Fig.10 The feature map fa5 shown in the figure obtains the first tampering edge map, wherein the number of layers of the target first extraction layer in the main branch network is not higher than the preset number of layers.

[0170] In step S460, a first segmentation map is obtained based on the first tampering edge map and one or more first feature maps through the first region detection network, wherein the first segmentation map indicates the suspected tampering area predicted by the tampering detection model. The implementation principle of step S460 is similar to that of the aforementioned step 45, and its implementation process can refer to the implementation process of the aforementioned step 45, which will not be described in detail here.

[0171] In some possible examples, the first area detection network includes: a plurality of first detection sub-networks connected in series, and the first segmentation map may be a segmentation map output by the last first detection sub-network.

[0172] In step S470, based at least on the first detection result and the first segmentation map, it is determined whether the card image contains a tampering detection result of the tampering area.

[0173] Understandably, Figure 4 The steps shown are only a schematic diagram of the tamper detection process and do not limit the execution order of the steps. In some implementations, steps S450-S460 may be executed before S430-S440, or may be executed in parallel with S430-S440.

[0174] In some possible examples, the electronic device can directly determine whether the first segmented image contains an image area with a pixel value of the first value from the perspective of the global card image, whether it contains a tampered area, and then determine whether the card image contains a tampered area in combination with the first detection result determined from the perspective of the local detail area of ​​the card image, so as to better ensure the accuracy of the determined tampering detection result. Exemplarily, if the first segmented image contains an image area with a pixel value of the first value, and the number of pixel values ​​of the first value in the image area is not less than a preset number, then from the perspective of the global card image, it can be determined that the tampered area is contained; otherwise, from the perspective of the global card image, it is determined that the first segmented image does not contain a tampered area.

[0175] In the latter implementation, if the first detection result indicates that the target area is a tampered area and / or it is determined from the global perspective of the card image that it contains a tampered area, then the tampering detection result can be determined as the card image contains a tampered area; if the first detection result indicates that the target area is not a tampered area, and it is determined from the global perspective of the card image that it does not contain a tampered area, then the tampering detection result is determined as the card image does not contain a tampered area. This can improve the detection rate of tampered areas in the card image, so as to better ensure the security of the online verification system.

[0176] In some possible examples, the tampering detection model obtained by the aforementioned training also includes a tampering region classification network. In step S470, steps 91-92 may be included:

[0177] In step 91, the tampering region classification network in the tampering detection model is used to determine a second detection result based on at least the first segmentation map. The process of obtaining the second detection result indicating whether the card image contains a tampering region based on the first segmentation map using the tampering region classification network can be referred to the implementation process of the aforementioned step 51, which will not be described in detail here.

[0178] In some possible examples, in step 91, the electronic device may use the tampered region classification network to determine the second detection result based on the first tampered edge map and the first segmentation map. The process of using the tampered region classification network to jointly determine the second detection result based on the first tampered edge map and the first segmentation map can refer to the implementation process of step 61 in the above embodiment, which will not be repeated here.

[0179] After the second detection result is obtained, at step 92 , a tampering detection result is determined based on the first detection result and the second detection result.

[0180] In some examples, the second detection result may be a score value indicating whether the card image contains a tampered area, and the first detection result may be a score value indicating whether the corresponding target area is a tampered area; in some implementations, the process of determining the tampering detection result may be: if at least one of the values ​​of the first detection result and the second detection result is greater than a preset threshold, it may be determined that the tampering detection result indicates that the card image contains a tampered area; otherwise, it may be determined that the tampering detection result indicates that the card image does not contain a tampered area. It may also be: in the case of multiple first detection results, the first detection result with the largest value is determined from the multiple first detection results; the first detection result with the largest value is weighted and summed with the second detection result to obtain a final detection result, which includes the final score value obtained by weighted summing the first detection result with the largest value and the second detection result; if the final score value is greater than the preset threshold, it may be determined that the tampering detection result indicates that the card image contains a tampered area; otherwise, it may be determined that the tampering detection result indicates that the card image does not contain a tampered area, so as to better improve the accuracy of the determined tampering detection result.

[0181] The above process combines the detection results of the local key object area in the card image and the global angle detection results of the card image to jointly determine the tampering detection results, which can not only improve the accuracy of the tampering detection results, but also improve the detection rate of tiny tampered areas in the card image.

[0182] In some other possible examples, in order to obtain a more accurate tampering detection result of whether the card image contains a tampered area, the electronic device may also combine multiple first feature maps containing global features, a first tampered edge map containing edge features, and a first segmentation map containing pixel features to jointly determine the second detection result. Exemplarily, the tampered area classification network may include a first classification subnetwork, a second classification subnetwork, and a third classification subnetwork; accordingly, in step S470, steps 101-104 may be included:

[0183] In step 101, a first result is determined through the aforementioned third classification sub-network based on a fourth feature map corresponding to the last extraction layer of the main branch network among the plurality of first feature maps.

[0184] In some examples, the electronic device can determine the fourth feature map corresponding to the last extraction layer of the main branch network from the multiple first feature maps, and then input the fourth feature map into the third classification subnetwork to obtain a first result, which is used to indicate whether the card image contains a tampered area. In some possible examples, the first result includes a first sub-value indicating that the card image contains a tampered area, wherein the higher the first sub-value, the greater the possibility that the card image includes a tampered area.

[0185] Among them, Fig.10 As shown, the third classification sub-network can be set after the main branch network, and its input can be the output corresponding to the last extraction layer of the main branch network.

[0186] In some other examples, the electronic device may also jointly determine the first result based on the fourth feature map and at least one other feature map in the plurality of first feature maps during the process of determining the aforementioned first result. Specifically, the electronic device may input the fourth feature map and at least one other feature map in the plurality of first feature maps into the third classification subnetwork, so as to fuse the fourth feature map and at least one other feature map in the plurality of first feature maps through the third classification subnetwork to obtain a fused feature, and then process the fused feature through the third classification subnetwork to obtain the first result.

[0187] In step 102, a second result is determined based on the first tampering edge graph through the aforementioned second classification sub-network.

[0188] In some examples, the electronic device may determine a second result based on the first tampering edge graph through a second classification subnetwork, the second result being used to indicate whether the card image contains a tampered area. Exemplarily, the second result includes a second subvalue indicating whether the card image contains a tampered area, wherein the higher the second subvalue, the greater the possibility that the card image contains a tampered area.

[0189] like Fig.10 As shown, the second classification subnetwork can be set after the first edge detection network, and its input is the output corresponding to the first edge detection network (for example, the first tampered edge map).

[0190] In step 103, the third result is determined based on the first segmentation map by the aforementioned first classification subnetwork. Exemplarily, the third result includes a third subvalue indicating whether the card image contains a tampered area, wherein the higher the third subvalue, the greater the possibility that the card image contains a tampered area.

[0191] like Fig.10 As shown, the first classification subnetwork can be set after the first area detection network, and its input is the output corresponding to each first detection subnetwork in the first area detection network, or it can be the output corresponding to the last first detection subnetwork in the first area detection network.

[0192] In step 104, the first result, the second result, the third result and the first detection result are combined to determine the tampering detection result. The implementation principle of combining the first result, the second result, the third result and the first detection result to determine the tampering detection result is similar to the implementation principle of determining the tampering detection result based on the first detection result and the second detection result. The implementation process can refer to the process of determining the tampering detection result based on the first detection result and the second detection result, which will not be described in detail here.

[0193] In some possible examples, when a user is handling a designated business online and needs to use his or her card to verify his or her identity, the user can upload an image containing his or her card image, i.e., the card image to be detected. After the electronic device obtains the card image, it needs to use the tampering detection method provided in this embodiment to detect tampering on the card image, i.e., detect whether it contains a tampered area. If it is determined that the card image does not contain a tampered area, the relevant processing procedures for the subsequent designated business can continue to be executed; if it is determined that the card image contains a tampered area, the electronic device can send a prompt message and can stop the relevant processing procedures for the subsequent designated business.

[0194] It is understandable that in the process of training the tampering detection model, the reversible branch network is used to assist the main branch network to learn more information in the image and ensure the accuracy of the detection results. Accordingly, in the process of tampering detection, in order to improve the detection efficiency, the main branch network, the first edge detection network and the first area detection network in the tampering detection model can be used to jointly execute the tampering detection process to obtain a detection result with higher accuracy. In addition, in the above process, the edge features in the image to be detected are enhanced based on the first edge detection network and the first area detection network, and a multi-channel information joint decision-making mechanism is introduced to better improve the accuracy of tampering area detection.

[0195] The foregoing describes certain embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0196] Corresponding to the above method embodiment, the present specification embodiment provides a tamper detection device 1100, whose schematic block diagram is as follows: Fig.11As shown, it includes: a first acquisition module 1110, configured to obtain the position information of the target area in the card image to be detected, wherein the target area is the area where the key object is located in the card image; a first acquisition module 1120, configured to extract features of the card image through one or more first extraction layers to obtain one or more first feature maps; a first determination module 1130, configured to determine the regional feature map corresponding to the target area based on the one or more first feature maps and the position information; a second acquisition module 1140, configured to obtain a first detection result of whether the target area is a tampered area based on the regional feature map through a first classification network; a second determination module 1150, configured to determine whether the card image contains a tampered area based on at least the first detection result.

[0197] In some possible examples, the region where the key object is located includes a region where a specified character string is located and / or a region where a face image is located.

[0198] In some possible examples, the first acquisition module 1110 is specifically configured to detect the card image based on a preset detection algorithm to obtain the position information of each candidate target area in the card image; use a specified area alignment algorithm to obtain a feature map mapped to a first size corresponding to each candidate target area based on the position information of each candidate target area; through a second classification network, based on the feature map of the first size corresponding to each candidate target area, determine whether each candidate target area is a classification result of the target area; based on the classification result, determine the position information of the target area from the position information of each candidate target area.

[0199] In some possible examples, the one or more first abstraction layers belong to a tampering detection model; the tampering detection model further includes a first edge detection network and a first area detection network;

[0200] The apparatus further includes: a third obtaining module (not shown in the figure), configured to obtain a first tampering edge map based on the one or more first feature maps through the first edge detection network; a fourth obtaining module (not shown in the figure), configured to obtain a first segmentation map based on the first tampering edge map and the one or more first feature maps through the first region detection network, wherein the first segmentation map indicates a suspected tampering region predicted by the tampering detection model;

[0201] The second determination module 1150 is specifically configured to determine whether the card image includes a tampering detection result of a tampered area based on at least the first detection result and the first segmentation map.

[0202] In some possible examples, the second determination module 1150 is specifically configured to use the tampering region classification network in the tampering detection model to determine a second detection result based at least on the first segmentation map;

[0203] The tampering detection result is determined based on the first detection result and the second detection result.

[0204] In some possible examples, the second obtaining module 1140 is specifically configured to process the regional feature map using a specified regional alignment algorithm to obtain a feature map mapped to a second size corresponding to the regional feature map;

[0205] Through the first classification network, based on the feature map of the second size corresponding to the regional feature map, a first detection result of whether the target area is a tampered area is obtained.

[0206] In some possible examples, there are multiple first feature maps;

[0207] The first determination module 1130 is specifically configured to fuse at least two specified first feature maps among the multiple first feature maps to obtain a fused feature map; based on the position information and the size ratio between the fused feature map and the card image, determine the regional feature map corresponding to the target area from the fused feature map.

[0208] In some possible examples, the tampering detection model further includes a reversible branch network; the reversible branch network includes a plurality of second extraction layers connected in series; the device further includes: a second acquisition module (not shown in the figure), configured to acquire a first sample image and its first label data, wherein the first label data is used to indicate whether the first sample image contains a tampered area;

[0209] A fifth obtaining module (not shown in the figure) is configured to perform feature extraction on the first sample image based on a plurality of first extraction layers connected in series in the main branch network to obtain a plurality of second feature maps corresponding to a plurality of specified first extraction layers;

[0210] a processing module (not shown in the figure), configured to process the first sample image based on the reversible branch network to obtain a plurality of third feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes, through a third extraction layer among the plurality of second extraction layers, performing feature extraction on a third feature map corresponding to a previous layer and a second feature map set corresponding to the third extraction layer to obtain an output feature map thereof, and obtaining a corresponding third feature map based on the output feature map and a second feature map of a first extraction layer at the same layer thereof, wherein the third extraction layer is a specified layer among the plurality of second extraction layers starting from the second second extraction layer;

[0211] a sixth obtaining module (not shown in the figure), configured to obtain a second tampering edge map based on a plurality of third feature maps through the first edge detection network;

[0212] a seventh obtaining module (not shown in the figure), configured to obtain a plurality of second segmentation maps based on a plurality of third feature maps and the second tampering edge map through the first region detection network;

[0213] A training module (not shown in the figure) is configured to train the tampering detection model based on each second segmentation map and the first label data.

[0214] In some possible examples, the tampering detection model further includes the first classification network; the first label data includes a label map generated based on the first sample image, label position information of an area where each sample key object is located in the first sample image, and label data indicating whether the area where each sample key object is located is a tampered area;

[0215] The device further includes: a third determination module (not shown in the figure), configured to determine a sample area feature map corresponding to an area where each sample key object is located based on the plurality of second feature maps and the label position information;

[0216] An eighth obtaining module (not shown in the figure) is configured to obtain prediction result data of whether the area where the key object of each sample is located is a tampered area based on the feature map of each sample area through the first classification network;

[0217] The training module is specifically configured to train the tampering detection model based on each second segmentation map and the label map, as well as the prediction result data and label data indicating whether the area where each sample key object is located is a tampered area.

[0218] In some possible examples, the first label data includes a label tampering edge map and a label segmentation map generated based on the first sample image; the training module is specifically configured to train the tampering detection model based on the second tampering edge map, each second segmentation map and the label tampering edge map and label segmentation map in the first label data.

[0219] In some possible examples, the first label data includes a label value indicating whether the first sample image includes a tampered area; the tampering detection model also includes a tampered area classification network;

[0220] The training module is specifically configured to utilize the tampering region classification network to determine a prediction value indicating whether the first sample image contains a tampering region based on the target segmentation image in each second segmentation image and the second tampering edge map; and train the tampering detection model based on the difference between the label value and the prediction value.

[0221] In some possible examples, the tampering detection model also includes a second edge detection network and a second area detection network, and the device also includes: an eighth obtaining module (not shown in the figure), configured to obtain a third tampering edge map based on multiple second feature maps through the second edge detection network; a ninth obtaining module (not shown in the figure), configured to obtain multiple third segmentation maps based on the third tampering edge map and multiple second feature maps through the second area detection network; the training module is specifically configured to train the tampering detection model based on each second segmentation map, each third segmentation map and the first label data.

[0222] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0223] The embodiments of the present specification also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the tampering detection method provided in the present specification.

[0224] An embodiment of the present specification further provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the tampering detection method provided in the present specification is implemented.

[0225] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium and computing device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0226] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0227] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A tampering detection method, comprising: Acquire location information of a target area in the card image to be detected, wherein the target area is an area where a key object is located in the card image; Extract features from the card image through one or more first extraction layers to obtain one or more first feature maps; Determine a region feature map corresponding to the target region based on the one or more first feature maps and the location information; Obtaining, by a first classification network, a first detection result of whether the target area is a tampered area based on the area feature map; At least based on the first detection result, it is determined whether the card image contains a tampering detection result of a tampered area.

2. The method of claim 1, wherein: The area where the key object is located includes the area where the specified character string is located and / or the area where the face image is located.

3. The method of claim 1, wherein: The step of obtaining the location information of the target area specified in the card image to be detected includes: Based on a preset detection algorithm, the card image is detected to obtain position information of each candidate target area in the card image; Using a specified region alignment algorithm, based on the position information of each candidate target region, a feature map corresponding to each candidate target region and mapped to the first size is obtained; Determining, by a second classification network, whether each candidate target region is a classification result of the target region based on a feature map of the first size corresponding to each candidate target region; Based on the classification result, the location information of the target area is determined from the location information of each candidate target area.

4. The method of claim 1, wherein: The one or more first abstraction layers belong to a tampering detection model; the tampering detection model also includes a first edge detection network and a first area detection network; The method further comprises: Obtaining, by the first edge detection network, a first tampering edge map based on the one or more first feature maps; Obtaining, by the first region detection network, a first segmentation map based on the first tampering edge map and the one or more first feature maps, wherein the first segmentation map indicates a suspected tampering region predicted by the tampering detection model; The determining, at least based on the first detection result, whether the card image contains a tampering detection result of a tampered area includes: A tampering detection result is determined based at least on the first detection result and the first segmentation map, indicating whether the card image contains a tampered area.

5. The method of claim 4, wherein: The determining whether the card image contains a tampering detection result of the tampering area includes: Determine a second detection result based on at least the first segmentation map using a tampering region classification network in the tampering detection model; The tampering detection result is determined based on the first detection result and the second detection result.

6. The method of claim 1, wherein: The obtaining, through the first classification network and based on the regional feature map, a first detection result of whether the target area is a tampered area includes: Using a specified region alignment algorithm, the region feature map is processed to obtain a feature map mapped to a second size corresponding to the region feature map; Through the first classification network, based on the feature map of the second size corresponding to the regional feature map, a first detection result of whether the target area is a tampered area is obtained.

7. The method of claim 1, wherein: There are multiple first feature graphs; The determining of the region feature map corresponding to the target region includes: Fusing at least two designated first feature maps among the plurality of first feature maps to obtain a fused feature map; Based on the position information and the size ratio between the fused feature map and the card image, a regional feature map corresponding to the target area is determined from the fused feature map.

8. The method of claim 4, wherein: The tamper detection model further includes a reversible branch network; the reversible branch network includes a plurality of second extraction layers connected in series; The method further comprises: Acquire a first sample image and first label data thereof, wherein the first label data is used to indicate whether the first sample image contains a tampered area; Perform feature extraction on the first sample image based on a plurality of first extraction layers connected in series in the main branch network to obtain a plurality of second feature maps corresponding to a plurality of specified first extraction layers; Processing the first sample image based on the reversible branch network to obtain a plurality of third feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes, through a third extraction layer among the plurality of second extraction layers, performing feature extraction on a third feature map corresponding to a previous layer and a second feature map set corresponding to the third extraction layer to obtain an output feature map thereof, and obtaining a corresponding third feature map based on the output feature map and a second feature map of a first extraction layer at the same layer thereof, wherein the third extraction layer is a specified layer among the plurality of second extraction layers starting from the second second extraction layer; Obtaining a second tampering edge map based on a plurality of third feature maps through the first edge detection network; Obtaining a plurality of second segmentation maps based on a plurality of third feature maps and the second tampering edge map through the first region detection network; The tampering detection model is trained based on each second segmentation map and the first label data.

9. The method of claim 8, wherein: The tampering detection model also includes the first classification network; the first label data includes a label map generated based on the first sample image, label position information of the area where each sample key object is located in the first sample image, and label data indicating whether the area where each sample key object is located is a tampered area; The method further comprises: Based on the multiple second feature maps and the label position information, determine the sample area feature map corresponding to the area where each sample key object is located; Obtain prediction result data of whether the area where the key object of each sample is located is a tampered area based on the feature map of each sample area through the first classification network; The training of the tampering detection model comprises: The tampering detection model is trained based on each second segmentation map and the label map, as well as the prediction result data and the label data indicating whether the area where each sample key object is located is a tampered area.

10. The method of claim 8, wherein: The first label data includes a label tampering edge map and a label segmentation map generated based on the first sample image; The training of the tampering detection model comprises: The tampering detection model is trained based on the second tampering edge map, each second segmentation map, and the label tampering edge map and label segmentation map in the first label data.

11. The method of claim 8, wherein the first label data comprises a label value for indicating whether the first sample image contains a tampered area; The tamper detection model also includes a tamper region classification network; The training of the tampering detection model comprises: Determining, using the tampered region classification network, a prediction value indicating whether the first sample image includes a tampered region based on a target segmented image in each of the second segmented images and the second tampered edge map; The tamper detection model is trained based on the difference between the label value and the predicted value.

12. The method of claim 8, wherein the tamper detection model further comprises a second edge detection network and a second area detection network, and the method further comprises: Obtaining a third tampering edge map based on the plurality of second feature maps through the second edge detection network; Obtaining, by the second region detection network, a plurality of third segmentation maps based on the third tampering edge map and a plurality of second feature maps; The training of the tampering detection model comprises: The tampering detection model is trained based on each second segmentation map, each third segmentation map and the first label data.

13. A tamper detection device, comprising: A first acquisition module is configured to acquire position information of a target area in a card image to be detected, wherein the target area is an area where a key object is located in the card image; A first obtaining module is configured to extract features from the card image through one or more first extraction layers to obtain one or more first feature maps; A first determining module, configured to determine a regional feature map corresponding to the target area based on the one or more first feature maps and the location information; A second obtaining module is configured to obtain a first detection result of whether the target area is a tampered area based on the area feature map through a first classification network; The second determination module is configured to determine whether the card image contains a tampering detection result of a tampered area based at least on the first detection result.

14. A computing device comprising a memory and a processor, wherein: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 12 is implemented.

Images