Tamper detection model training method and device and tamper detection method and device

By combining the main branch network and the reversible branch network in the card certificate image detection model, the problem of inaccurate detection in the prior art is solved, the accuracy of identification of tampering areas is improved, and the online collaborator is enhanced Verify the safety of the system.

CN119027758APending Publication Date: 2024-11-26ANT BLOCKCHAIN TECHNOLOGY (SHANGHAI) CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411104809.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-12
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The prior art is difficult to effectively detect and identify tampered areas in card certificate images, which leads to the threat of the security of the online verification system.

Method used

A tamper detection model including a main branch network, a reversible branch network, a first edge detection network and a first area detection network are used to identify the tamper region in the image through feature extraction and edge information processing.

Benefits of technology

It improves the accurate detection and recognition ability of tampering areas of different sizes, enhances the accuracy of image tampering position detection, and enhances the security of the online verification system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027758A_ABST
    Figure CN119027758A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a tampering detection model training method and device and a tampering detection method and device.The model comprises a main branch network, a reversible branch network, a first edge detection network and a first region detection network, and the training method comprises the steps that a sample image is processed through the main branch network, and a first feature map of a specified first extraction layer is obtained; processing the sample image through a reversible branch network to obtain a second feature map of a specified second extraction layer, and during processing, combining the second feature map of the previous layer, the first feature map corresponding to the second feature map and the first feature map of the first extraction layer of the same layer through a specified third extraction layer to obtain a corresponding second feature map; processing the plurality of second feature maps through a first edge detection network to obtain a first tampered edge map; processing the plurality of second feature maps and the first tampering edge map through a first region detection network to obtain a plurality of first segmentation maps; and training the model based on each first segmentation image and the label data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a tampering detection model training, tampering detection method and device. Background Art

[0002] In recent years, with the rapid development of the Internet, people can handle various businesses online and verify relevant cards and certificates (including bank cards, identity documents, etc.) online. The online mode is different from the offline mode. The online verification method that only verifies through card and certificate images puts the verification security under great test.

[0003] At present, there are some people who tamper with cards (bank cards, identity documents, etc.) (for example, physically cover the information of the card itself through paper strips and / or other facial images, handwriting pollution, etc., or modify the card information in the card image through image modification technology) in order to deceive the online verification system of relevant institutions. The above-mentioned behavior puts the security of the online verification system of relevant institutions under great test. In view of this, it is very important to provide a training method for a tampering detection model for detecting tampered areas in card images. Summary of the invention

[0004] One or more embodiments of the present specification provide a tampering detection model training, tampering detection method and device to achieve accurate detection of tampering locations in an image.

[0005] According to a first aspect, a training method for a tampering detection model is provided, wherein the tampering detection model includes a main branch network, a reversible branch network, a first edge detection network, and a first area detection network, wherein the main branch network includes a plurality of first extraction layers connected in series, and the reversible branch network includes a plurality of second extraction layers connected in series, and the method includes:

[0006] Acquire a first sample image and first label data thereof, wherein the first label data is used to indicate whether the first sample image contains a tampered area;

[0007] Performing first feature extraction on the first sample image based on the main branch network to obtain a plurality of first feature maps corresponding to a plurality of specified first extraction layers;

[0008] Processing the first sample image based on the reversible branch network to obtain a plurality of second feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes, through a third extraction layer among the plurality of second extraction layers, performing second feature extraction on the second feature map corresponding to the previous layer and the first feature map set corresponding to the third extraction layer to obtain its output feature map, and obtaining the corresponding second feature map based on the output feature map and the first feature map of the first extraction layer at the same layer, wherein the third extraction layer is a specified layer among the plurality of second extraction layers starting from the second second extraction layer;

[0009] Obtaining a first tampering edge map based on a plurality of second feature maps through the first edge detection network;

[0010] Obtaining a plurality of first segmentation maps based on a plurality of second feature maps and the first tampering edge map through the first region detection network;

[0011] The tampering detection model is trained based on each first segmentation map and the first label data.

[0012] According to a second aspect, a tampering detection method is provided, comprising:

[0013] Acquire the training method of the tampering detection model based on the first aspect, the main branch network, the first edge detection network and the first area detection network in the tampering detection model obtained by training;

[0014] Acquire the image to be detected containing the card to be detected;

[0015] Based on the main branch network, performing first feature extraction on the image to be detected to obtain a plurality of third feature maps corresponding to a plurality of specified first extraction layers;

[0016] Obtaining a third tampering edge map based on a plurality of third feature maps through the first edge detection network;

[0017] Obtaining, by the first region detection network, a third segmentation map based on the third tampering edge map and a plurality of third feature maps;

[0018] A detection result of whether the card to be detected contains a tampered area is determined based at least on the third segmentation map.

[0019] According to a third aspect, a training device for a tampering detection model is provided, wherein the tampering detection model includes a main branch network, a reversible branch network, a first edge detection network, and a first area detection network, wherein the main branch network includes a plurality of first extraction layers connected in series, and the reversible branch network includes a plurality of second extraction layers connected in series, and the device includes:

[0020] A first acquisition module is configured to acquire a first sample image and first label data thereof, wherein the first label data is used to indicate whether the first sample image contains a tampered area;

[0021] A first extraction module is configured to perform first feature extraction on the first sample image based on the main branch network to obtain a plurality of first feature maps corresponding to a plurality of specified first extraction layers;

[0022] A first processing module is configured to process the first sample image based on the reversible branch network to obtain a plurality of second feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes, through a third extraction layer among the plurality of second extraction layers, performing second feature extraction on a second feature map corresponding to a previous layer and a first feature map set corresponding to the third extraction layer to obtain an output feature map thereof, and obtaining a corresponding second feature map based on the output feature map and a first feature map of a first extraction layer at the same layer thereof, wherein the third extraction layer is a specified layer among the plurality of second extraction layers starting from the second second extraction layer;

[0023] A first obtaining module is configured to obtain a first tampering edge map based on a plurality of second feature maps through the first edge detection network;

[0024] a second obtaining module configured to obtain a plurality of first segmentation maps based on a plurality of second feature maps and the first tampering edge map through the first region detection network;

[0025] A training module is configured to train the tampering detection model based on each first segmentation map and the first label data.

[0026] According to a fourth aspect, there is provided a tamper detection device, comprising:

[0027] A second acquisition module is configured to acquire a training device based on the tampering detection model according to the third aspect, a main branch network, a first edge detection network, and a first area detection network in the trained tampering detection model;

[0028] A third acquisition module is configured to acquire an image to be detected containing a card to be detected;

[0029] A second extraction module is configured to perform first feature extraction on the image to be detected based on the main branch network to obtain a plurality of third feature maps corresponding to a plurality of specified first extraction layers;

[0030] A third obtaining module is configured to obtain a third tampering edge map based on a plurality of third feature maps through the first edge detection network;

[0031] a fourth obtaining module configured to obtain a third segmentation map based on the first tampering edge map and a plurality of third feature maps through the first region detection network;

[0032] The determination module is configured to determine, based at least on the third segmentation graph, whether the card to be detected contains a detection result of a tampered area.

[0033] According to a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in the first aspect or the second aspect.

[0034] According to a sixth aspect, a computing device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in the first aspect or the second aspect is implemented.

[0035] According to the training of the tampering detection model, the tampering detection method and the device provided in the embodiments of the present specification, wherein the tampering detection model includes a main branch network including multiple first extraction layers connected in series, a reversible branch network including multiple second extraction layers connected in series, a first edge detection network and a first area detection network. During the training process, the main branch network and the reversible branch network are combined to perform feature extraction on the first sample image to obtain multiple first feature maps corresponding to multiple specified first extraction layers and multiple second feature maps corresponding to multiple specified second extraction layers. For the specified third extraction layer in the reversible branch network, the second feature map corresponding to the previous layer and the first feature map corresponding to it are combined with the first feature map of the first extraction layer of the same layer to obtain The corresponding second feature map that integrates more image information (including global information and local detail information) is combined with a reversible branch network and multi-level auxiliary information, namely the first area detection network. Through the guidance of the first tampering edge map, based on multiple second feature maps, multiple first segmentation maps corresponding to multiple different extraction layers with more edge information and higher accuracy are obtained. Among them, the smaller the number of layers of the second extraction layer in the reversible branch network, the more local and detail image information its second feature map contains, and accordingly it is more conducive to the detection of relatively small tampering areas. Subsequently, the first segmentation maps and the first label data are used to train the tampering detection model, so as to improve the tampering detection model's accurate detection and identification of tampering areas of different sizes, and improve the accuracy of the image tampering position (area) detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1A A schematic diagram of an implementation framework of an embodiment disclosed in this specification;

[0038] Figure 1B An exemplary card image including a tampered area provided for this specification;

[0039] Figure 1C An exemplary card image including a tampered area provided for this specification;

[0040] Figure 1D An exemplary card image including a tampered area provided for this specification;

[0041] Figure 2 A schematic diagram of a flow chart of a training method for a tamper detection model provided in an embodiment;

[0042] Figure 3A A schematic diagram of a structure of a tamper detection model provided in an embodiment;

[0043] Figure 3B A schematic diagram of the structure of a first edge detection network provided in an embodiment;

[0044] Figure 3C A schematic diagram of the structure of a first edge enhancement sub-network provided in an embodiment;

[0045] Figure 3D A schematic diagram of another structure of a tamper detection model provided in an embodiment;

[0046] Figure 4 A schematic diagram of another flow chart of a method for training a tamper detection model provided in an embodiment;

[0047] Figure 5 A schematic diagram of another structure of a tamper detection model provided in an embodiment;

[0048] Figure 6 A schematic diagram of a flow chart of a tampering detection method provided in an embodiment;

[0049] Figure 7 A schematic diagram of a tampering detection process provided by an embodiment;

[0050] Figure 8 A schematic block diagram of a training device for a tamper detection model provided in an embodiment;

[0051] Fig. 9 A schematic block diagram of a tamper detection device provided in an embodiment. DETAILED DESCRIPTION

[0052] The technical solutions of the embodiments of this specification will be described in detail below with reference to the accompanying drawings.

[0053] The embodiments of this specification disclose a tampering detection model training, tampering detection method and device. The following first introduces the application scenarios and technical concepts of the method, which are as follows:

[0054] As mentioned above, at present, there are some people who tamper with cards (bank cards, identity documents, etc.) (for example, by physically covering the information of the card itself with paper slips and / or other facial images, handwriting pollution, etc.) in order to deceive the online verification system of relevant institutions, which greatly tests the security of the online verification system of relevant institutions. In view of this, it is very important to provide a training method for a tampering detection model for detecting tampered areas in card images.

[0055] In view of this, the inventor proposes a training method for a tampering detection model and a tampering detection method. Figure 1A A schematic diagram of an implementation scenario according to an embodiment disclosed in this specification is shown. In this implementation scenario, the tampering detection model includes a main branch network, a reversible branch network, a first edge detection network and a first region detection network. The main branch network includes a plurality of first extraction layers connected in series, and the reversible branch network includes a plurality of second extraction layers connected in series. The first edge detection network pays more attention to the edge information of the tampered area in the image, and is a network for detecting the edge information of the tampered area in the image; the first region detection network pays more attention to the overall regional structure information of the tampered area in the image, and is a network for detecting the location of the tampered area in the image.

[0056] During the training process, a training data set S for training the tampering detection model can be first constructed. The training data set S includes several sample images X and their label data Y. The sample images X can be card images containing tampered areas (i.e., the sample images contain tampered areas), or card images not containing tampered areas (i.e., the sample images do not contain tampered areas). The tampered areas can include the tampered areas in the cards (bank cards and identity documents) that are marked with paper strips (such as Figure 1B The area indicated by the arrow shown) and / or other face images, handwriting contamination (such as Figure 1C The area indicated by the arrow in the figure) is blocked and tampered by physical blocking tampering methods; it can also include the area modified by image modification technology in the card (such as Figure 1DThe information in the card on the left is the real information, and the information in the card on the right is the modified information).

[0057] In some examples, the label data corresponding to each sample image may include a label segmentation map generated based on the tampered area in the sample image; and may also include a label edge map generated based on the tampered area in the sample image.

[0058] Among them, the label segmentation map can be a binary mask image. For a sample image containing a tampered area, this type of binary mask image can indicate the location of the tampered area in the corresponding sample image. Exemplarily, the pixel value of the pixel indicating the tampered area is a first value (for example, 1), and the pixel value of the pixel indicating the non-tampered area is a second value (for example, 0). For a sample image that does not contain a tampered area, this type of binary mask image can indicate that the corresponding sample image does not contain a tampered area, and the pixel values ​​of the pixels can all be the second value.

[0059] The label edge map may also be a binary image. For a sample image containing a tampered area, this type of binary image may indicate edge information of the tampered area in the corresponding sample image. For example, the pixel values ​​of the pixels indicating the edge of the tampered area are the first value, and the pixel values ​​of the pixels at the edge of other non-tampered areas are the second value. For a sample image that does not contain a tampered area, this type of binary image may indicate that the corresponding sample image does not contain a tampered area, and the pixel values ​​of the pixels may all be the second value.

[0060] In some other examples, the label data corresponding to each sample image may also include a label value indicating whether the sample image contains a tampered area. For example, a label value of a first value (e.g., 1) indicates that the sample image includes a tampered area, and a label value of a second value (e.g., 0) indicates that the sample image does not include a tampered area.

[0061] After obtaining the training data set S, the tampering detection model can be trained based on each sample image and its label data in the training data set S. It can be understood that the process of training the tampering detection model based on each sample image and its label data is similar. The following takes any first sample image Xj and its first label data Yj in the training data set S as an example to introduce the process of training the tampering detection model. The process of training the tampering detection model using other sample images and their label data can refer to the process of training the tampering detection model using the first sample image and its first label data.

[0062] Specifically, an arbitrary first sample image Xj and its first label data Yj are obtained from a training data set S, wherein the first sample image is a card image containing a tampered area; the first sample image Xj is respectively input into a main branch network and a reversible branch network, and a first feature extraction is performed on the first sample image Xj based on a plurality of first extraction layers of the main branch network to obtain a plurality of first feature maps corresponding to a plurality of specified first extraction layers; the first sample image Xj is processed based on the reversible branch network to obtain a plurality of second feature maps corresponding to a plurality of specified second extraction layers, wherein the aforementioned processing includes, through a third extraction layer among the plurality of second extraction layers, performing a second feature extraction on a second feature map corresponding to a previous layer and a first feature map set corresponding to the third extraction layer to obtain an output feature map, and based on the output feature map and a first feature map of a first extraction layer at the same layer as the third extraction layer, obtaining a second feature map corresponding to the third extraction layer, wherein the third extraction layer is a specified layer among the plurality of second extraction layers starting from the second second extraction layer.

[0063] Among them, the third extraction layer can be one or more, and can be multiple extraction layers pre-designated among multiple second extraction layers. Exemplarily, it can be several second extraction layers with a larger number of layers among multiple second extraction layers; or several second extraction layers with a middle number of layers among multiple second extraction layers; or several second extraction layers with a smaller number of layers among multiple second extraction layers.

[0064] After obtaining multiple second feature maps, then, through the first edge detection network, based on the multiple second feature maps, a tampered edge map is predicted, called the first tampered edge map; through the first region detection network, based on the multiple second feature maps and the first tampered edge map, multiple first segmentation maps are obtained, wherein each first segmentation map includes the predicted tampered area in the first sample image; then, based on each first segmentation map and the first label data Yj, a tampering detection model is trained.

[0065] In the above process, for the third extraction layer in the reversible branch network, the second feature map corresponding to the previous layer and the first feature map corresponding to it are combined with the first feature map of the first extraction layer in the same layer to obtain the corresponding second feature map that integrates more image information (including global information and local detail information). The reversible branch network and the multi-level auxiliary information, i.e., the first area detection network, are combined. Through the guidance of the first tampering edge map, based on multiple second feature maps, multiple first segmentation maps corresponding to multiple different extraction layers with more edge information and higher accuracy are obtained. Among them, the smaller the number of layers of the second extraction layer in the reversible branch network, the more local and detail information of the image contained in its second feature map, and accordingly, it is more conducive to the detection of relatively small tampering areas. Subsequently, the first segmentation maps and the first label data are used to train the tampering detection model, so as to improve the tampering detection model's accurate detection and identification of tampering areas of different sizes, and improve the accuracy of the image tampering position (area) detection results.

[0066] In some examples, such as Figure 1A As shown, the first tampering edge map and each first segmentation map and the first label data (the label tampering edge map and the label segmentation map) can also be used to jointly train the tampering detection model. In some other examples, the first label data includes a label value indicating whether the first sample image includes a tampered area; the tampering detection model can also include a classification network, and accordingly, the classification network can also be used to obtain a predicted value of whether the predicted first sample image includes a tampered area using each first segmentation map (and the first tampering edge map); then the tampering detection model is trained using the predicted value and the label value so that the tampering detection model can detect whether the image includes a tampered area.

[0067] The following is a detailed description of the training method of the tampering detection model and the tampering detection method provided in this specification in conjunction with specific embodiments.

[0068] Figure 2 A flowchart of a method for training a tamper detection model in one embodiment of the present specification is shown. The method is executed by a first electronic device, and the first electronic device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. The tamper detection model includes a main branch network, a reversible branch network, a first edge detection network, and a first regional detection network. The main branch network includes a plurality of first extraction layers connected in series, and the reversible branch network includes a plurality of second extraction layers connected in series. During the training process of the tamper detection model, Figure 2 As shown, the method includes the following steps S210-S260:

[0069] In step S210, a first sample image and its first label data are obtained, wherein the first label data is used to indicate whether the first sample image contains a tampered area.

[0070] It can be understood that the first sample image and its first label data can be any sample image and its label data in the aforementioned training data set. The first label data may include a label segmentation map and a label edge map generated based on the first sample image. The label segmentation map can indicate whether the first sample image (i.e., the card certificate) contains a tampered area. If the first sample image contains a tampered area, the label segmentation map in its first label data can indicate the location of the tampered area contained in the first sample avatar. The label edge map can indicate whether the first sample image contains a tampered area. If the first sample image contains a tampered area, the label edge map in its first label data can indicate the edge position of the tampered area contained in the first sample image. Among them, the aforementioned card certificates may include but are not limited to: bank cards, identity documents, etc.

[0071] After the first electronic device obtains the first sample image and its first label data, in step S220, first feature extraction is performed on the first sample image based on the main branch network to obtain multiple first feature maps corresponding to multiple specified first extraction layers.

[0072] In this step, the first electronic device can input the first sample image into the main branch network, and perform first feature extraction on the first sample image based on the multiple first extraction layers connected in series in the main branch network to obtain multiple first feature maps corresponding to multiple specified first extraction layers. Among the multiple first extraction layers connected in series, the input of the first first extraction layer is the first sample image, and the input of each non-first first extraction layer is the output of the previous first extraction layer.

[0073] In some possible implementations, each first extraction layer of the main branch network may include but is not limited to operations for extracting features from an image, such as convolution operations, pooling operations, and batch normalization operations. In some examples, the main branch network may be an extraction network based on a Res2Net-50 structure to extract features from a first sample image, and obtain multiple first feature maps corresponding to multiple specified first extraction layers, and the sizes of the first feature maps corresponding to different specified first extraction layers may be different.

[0074] The aforementioned designated first extraction layer may at least include the first extraction layers corresponding to the third extraction layers in the reversible branch network mentioned later and the first extraction layers in the same layer as the third extraction layers. Figure 3A As shown, the main branch network may include 5 first extraction layers, such as Figure 3AThe extraction layers 11, 12, 13, 14 and 15 shown in FIG. 1 are shown in FIG. 1 . The designated first extraction layer may include various non-first-layer first extraction layers, for example, including Figure 3A The extraction layer 12, the extraction layer 13, the extraction layer 14 and the extraction layer 15 are shown.

[0075] Next, in step S230, the first sample image is processed based on the reversible branch network to obtain multiple second feature maps corresponding to multiple specified second extraction layers, wherein the processing includes, through a third extraction layer among the multiple second extraction layers, performing second feature extraction on the second feature map corresponding to its previous layer and the first feature map set corresponding to it to obtain its output feature map, and obtaining the corresponding second feature map based on the output feature map and the first feature map of the first extraction layer of the same layer, wherein the third extraction layer is a specified layer among the multiple second extraction layers starting from the second second extraction layer.

[0076] In this step, the first electronic device inputs the first sample image into the reversible branch network, and processes the first sample image based on the multiple second extraction layers connected in series included in the reversible branch network, wherein the multiple second extraction layers connected in series include one or more third extraction layers, and the input of each third extraction layer includes the second feature map corresponding to the previous extraction layer and the first feature map set corresponding to the third extraction layer; the input of the other second extraction layers except the third extraction layer in the multiple second extraction layers connected in series can be the first sample image or the second feature map corresponding to the previous extraction layer.

[0077] In some possible examples, for each third extraction layer in a plurality of second extraction layers connected in series (taking the third extraction layer P as an example for illustration), the third extraction layer P includes a fusion unit P1 and an extraction unit P2, and the aforementioned process of performing second feature extraction on the second feature map a1 corresponding to the previous layer and the first feature map b1 set to correspond to it through the third extraction layer to obtain its output feature map may include: respectively adjusting the sizes of the second feature map a1 corresponding to the previous extraction layer and the first feature map b1 set to correspond to the third extraction layer (for example, upsampling and / or downsampling) to unify the sizes of the second feature map a1 corresponding to the previous extraction layer and the first feature map b1 set to correspond to the third extraction layer.

[0078] Next, the fusion unit P1 performs a first preset processing on the resized second feature map a2 corresponding to the previous extraction layer of the third extraction layer to obtain the corresponding processed second feature map a3, and performs a second preset processing on the resized first feature map b2 corresponding to the third extraction layer to obtain the corresponding processed first feature map b3; the processed second feature map a3 and the processed first feature map b3 are first fused to obtain a fused feature map c of the third extraction layer; the extraction unit P2 performs feature extraction on the aforementioned fused feature map c to obtain the output feature map of the third extraction layer P.

[0079] For example, the first preset processing may include but is not limited to convolution operations and batch normalization operations, and the second preset processing may include but is not limited to linear processing, batch normalization operations, and difference processing, etc.; the aforementioned first fusion processing may include pixel-by-pixel addition of the processed second feature map and the processed third feature map, that is, adding the value of the pixel at the ath row and bth column position in the processed second feature map to the value of the pixel at the ath row and bth column position in the processed third feature map.

[0080] The aforementioned extraction unit P2 can be any extraction unit in the related art that can realize feature extraction, such as a feature extraction unit based on a residual network, or a feature extraction unit based on a ConvNeXt network, etc.

[0081] Next, after obtaining the output feature map through the third extraction layer P, a preset reversible processing is performed on the first feature map of the first extraction layer on the same layer as the third extraction layer P to obtain a feature map after reversible processing, and a second fusion processing is performed on the output feature map of the third extraction layer P and the feature map after reversible processing to obtain a second feature map corresponding to the third extraction layer P.

[0082] The preset reversible processing may refer to an operation in which the corresponding output can obtain the corresponding input through the inverse operation of the preset reversible processing. The reversible processing may include, but is not limited to, multiplying the first feature map of the first extraction layer at the same layer as the third extraction layer P by a preset coefficient; adding the first feature map of the first extraction layer at the same layer as the third extraction layer P to specified data, etc. The second fusion processing may be pixel-by-pixel addition of the output feature map of the third extraction layer P and the feature map after the reversible processing.

[0083] In some examples, the structure of the aforementioned other second extraction layer may be similar to the structure of the aforementioned third extraction layer, wherein the first feature map corresponding to it (the aforementioned other second extraction layer) may be empty. In some other examples, the structure of the aforementioned other second extraction layer may be similar to the structure of the first extraction layer.

[0084] In some possible examples, the number of the multiple second extraction layers of the reversible branch network may be the same as or different from the number of the multiple first extraction layers of the main branch network; when different, the number of the multiple second extraction layers of the reversible branch network may be less than the number of the multiple first extraction layers of the main branch network. Figure 3A As shown, the number of the second extraction layers of the reversible branch network can be the same as the number of the first extraction layers of the main branch network, both of which are 5 layers. Figure 3A As shown, the reversible branch network includes extraction layer 21, extraction layer 22, extraction layer 23, extraction layer 24 and extraction layer 25.

[0085] In some implementations, the third extraction layer may be set as required, for example, it may include one or more third extraction layers, and the third extraction layer may be an extraction layer with a higher number of layers in the reversible branch network, or an extraction layer with a lower number of layers in the reversible branch network. Figure 3A As shown, the third extraction layer may be the third extraction layer to the fifth extraction layer in the reversible branch network, namely, extraction layer 23, extraction layer 24 and extraction layer 25.

[0086] In some possible examples, the third extraction layer is the i-th extraction layer in the reversible branch network; the first feature map corresponding to the third extraction layer can be the first feature map of the i+n-th first extraction layer in the main branch network, where n is a positive integer. The specific value of n can be set according to requirements and the number of first extraction layers in the main branch network.

[0087] For example, n can be set to 1 or 2, etc. Figure 3A As shown, assuming that n is set to 1, for the extraction layer 23 in the reversible branch network, the first feature map corresponding to the third extraction layer is set to be the first feature map of the first extraction layer of the 3+nth layer (n is 1) in the main branch network, that is, the fourth first extraction layer in the main branch network (that is, Figure 3A For the extraction layer 24 in the reversible branch network, the first feature map corresponding to the third extraction layer is set to be the fifth first extraction layer in the main branch network (i.e., Figure 3A The first characteristic map of the extraction layer 15 shown; for the extraction layer 25 in the reversible branch network, the first characteristic map corresponding to the third extraction layer is empty.

[0088] The aforementioned multiple designated second extraction layers can be set according to requirements, and the multiple designated second extraction layers include, for example: the non-first layer second extraction layers in the multiple second extraction layers. Figure 3A The extraction layer 22, the extraction layer 23, the extraction layer 24 and the extraction layer 25 are shown.

[0089] After the first electronic device obtains a plurality of second feature maps corresponding to a plurality of specified second extraction layers, in step S240, a first tampering edge map is obtained based on the plurality of second feature maps through a first edge detection network.

[0090] In some possible examples, considering the second feature maps corresponding to different designated second extraction layers, the shallower the designated second extraction layer is in the reversible branch network, the more local edge detail information in the image contained in the corresponding second feature map, which can provide more edge information of the tampered area; the deeper the designated second extraction layer is in the reversible branch network, the more global information in the image (the location information of each tampered area therein) contained in the corresponding second feature map, that is, the supervision of the edge positioning of the tampered area can be better achieved. In order to enable the first edge detection network to predict the edge information of the tampered area with higher accuracy, the first electronic device can input the multiple second feature maps into the first edge detection network after obtaining the multiple second feature maps corresponding to the multiple designated second extraction layers, and obtain the first tampered edge map based on the multiple second feature maps through the first edge detection network, so as to better mine the edge information of the tampered area.

[0091] In some possible examples, based on the above situation and taking into account the model training efficiency, the aforementioned multiple second feature maps may include: a feature map corresponding to a target second extraction layer of the reversible branch network and a feature map corresponding to the last second extraction layer, and the number of layers of the target second extraction layer in the reversible branch network is not higher than a preset number of layers; accordingly, in step S240, the following steps are included: the feature map corresponding to the target second extraction layer and the feature map corresponding to the last second extraction layer are input into the first edge detection network to determine a first tampered edge map.

[0092] Among them, the number of layers of the target second extraction layer in the reversible branch network is not higher than the preset number of layers, and its corresponding feature map contains more local edge detail information in the image, and the feature map corresponding to the last second extraction layer contains more global information in the image. The first electronic device inputs the feature map corresponding to the target second extraction layer and the feature map corresponding to the last second extraction layer into the first edge detection network, so that the first edge detection network can mine out edge semantics related to the tampered area with better accuracy from the feature map corresponding to the target second extraction layer containing more local edge detail information in the image (also called low-level features) and the feature map corresponding to the last second extraction layer containing more global information under the supervision of the tampered area boundary (also called high-level features), thereby obtaining a first tampered edge map with higher accuracy.

[0093] In one implementation, the aforementioned target second extraction layer may be the second extraction layer in the reversible branch network, such as Figure 3A and Figure 3B As shown, the feature map corresponding to the target second extraction layer is feature map f22, and the feature map corresponding to the last second extraction layer is feature map f25.

[0094] Exemplarily, the first edge detection network may be any network model in the related art that can realize edge detection. In some possible examples, the structure of the first edge detection network may be as follows: Figure 3B As shown, it may include a first convolutional layer and a second convolutional layer arranged in parallel, wherein the input of the first convolutional layer is the feature map corresponding to the aforementioned target second extraction layer (e.g., feature map f12), and the input of the second convolutional layer is the feature map corresponding to the aforementioned last second extraction layer (e.g., feature map f15). The second convolutional layer is followed by an upsampling layer "U", which is then connected to a concatenation layer "C"; the first convolutional layer is connected to the concatenation layer; and the concatenation layer is followed by a plurality of third convolutional layers and an activation layer. The sizes of the convolution kernels of the first convolutional layer and the second convolutional layer may both be 1*1.

[0095] Exemplarily, the third convolutional layers connected to the concatenated layer may be one or more. In some possible examples, for example Figure 3B As shown, the plurality of third convolutional layers may include a plurality of serially connected convolutional layers, specifically including: a third convolutional layer 1 with two serially connected convolution kernels of size 3*3 and a third convolutional layer 2 with a convolution kernel size of 1*1. The aforementioned splicing layer is provided with a splicing function (for example, a concat function) to splice its two inputs. The activation layer is provided with an activation function (for example, a Sigmoid function). Through the above structure of the first edge detection network, based on the local edge detail information contained in the feature map corresponding to the target second extraction layer and the global information contained in the feature map corresponding to the last second extraction layer, a first tampered edge map with higher accuracy can be determined.

[0096] Next, in step S250, a plurality of first segmentation maps are obtained based on the plurality of second feature maps and the first tampering edge map through a first region detection network, wherein the first segmentation map includes the predicted tampering region in the first sample image.

[0097] In this step, the first electronic device can integrate the first tampered edge map with each second feature map, i.e., each level feature, through the first area detection network, and guide each level feature to learn the edge information therein through the first tampered edge map to enhance the edge information in the features of each level; and then obtain multiple first segmented images based on the features of each level after the edge information is enhanced through the first area detection network.

[0098] Some possible examples include Figure 3AAs shown, the first area detection network may include: a plurality of first detection sub-networks connected in series (including Figure 3A , "CAM1", "CAM2" and "CAM3" as shown), and the first edge enhancement sub-networks corresponding to the specified second extraction layers (including Figure 3A As shown, "EFM1" corresponding to the extraction layer 22 is input as the second feature map f22 corresponding to the extraction layer 22; "EFM2" corresponding to the extraction layer 23 is input as the second feature map f23 corresponding to the extraction layer 23; "EFM3" corresponding to the extraction layer 24 is input as the second feature map f24 corresponding to the extraction layer 24; "EFM4" corresponding to the extraction layer 25 is input as the second feature map f25 corresponding to the extraction layer 25); accordingly, step S250 may include the following steps 11-12:

[0099] In step 11, the second feature map and the first tampered edge map corresponding to each first edge enhancement sub-network are processed respectively to obtain the edge enhancement feature map of each first edge enhancement sub-network. It can be understood that the second extraction layer is specified to correspond to the first edge enhancement sub-network, the second extraction layer is specified to correspond to the second feature map, and accordingly, the first edge enhancement sub-network corresponds to the second feature map.

[0100] In this step, the first electronic device combines the second feature maps corresponding to each specified second extraction layer with the first tampered edge map respectively to obtain multiple image groups, each image group includes a second feature map corresponding to a specified second extraction layer and a first tampered edge map; each image group is input into each first edge enhancement sub-network corresponding to each specified second extraction layer, so that each first edge enhancement sub-network processes its corresponding second feature map and the first tampered edge map respectively, and integrates the edge information of the first tampered edge map with the features in the corresponding second feature map, that is, injects the edge information of the first tampered edge map into the representation learning of the corresponding second feature map to enhance the edge feature representation in the corresponding second feature map, and obtains the edge enhanced feature map of each first edge enhancement sub-network.

[0101] In some possible examples, the first edge enhancement sub-network may be any neural network that can realize feature fusion integration. Exemplarily, in order to achieve good integration and obtain edge enhancement feature maps of each first edge enhancement sub-network with better edge enhancement effect, the first edge enhancement sub-network may introduce a local attention mechanism, and utilize the introduced local attention mechanism to perform feature interaction and integration between the corresponding second feature map and the first tampered edge map. Figure 3CAs shown, it is a structural schematic diagram of an arbitrary first edge enhancement sub-network k (k is a positive integer), which includes a downsampling layer, a first fusion layer, a second fusion layer, a fifth convolutional layer, a processing layer based on a local attention mechanism, a third fusion layer and a sixth convolutional layer connected in sequence.

[0102] like Figure 3C As shown, the input of the downsampling layer is the first tampered edge map; the first fusion layer is set to pixel-by-pixel multiplication processing, and its input is the corresponding second feature map and the first tampered edge map after downsampling by the downsampling layer; the second fusion layer is set to pixel-by-pixel addition processing, and its input is the corresponding second feature map and the output of the first fusion layer; the input of the fifth convolution layer is the output of the second fusion layer, and the size of its convolution kernel is set to 3*3, for example; the processing layer based on the local attention mechanism may include a pooling sublayer (for example, a GAP (Global Average Pooling) layer), a convolution sublayer and an activation sublayer (in which, for example, a Sigmoid activation function is set); the third fusion layer is set to pixel-by-pixel multiplication, and its input is the output of the fifth convolution layer and the output of the processing layer based on the local attention mechanism; the input of the sixth convolution layer is the output of the third fusion layer.

[0103] The first electronic device obtains the edge enhancement feature map of each first edge enhancement subnetwork, and then, in step 12, processes the edge enhancement feature map and the corresponding first input data of the first edge enhancement subnetwork corresponding to it through any target detection subnetwork among the multiple first detection subnetworks to obtain the first segmentation map corresponding to the target detection subnetwork, wherein, when the target detection subnetwork is the first first detection subnetwork, its corresponding first input data is the edge enhancement feature map of the first edge enhancement subnetwork specified among the multiple first edge enhancement subnetworks; when the target detection subnetwork is not the first first detection subnetwork, its corresponding first input data is the output of the previous first detection subnetwork.

[0104] In this step, the first electronic device can use each first detection subnetwork as a target detection subnetwork in turn according to the serial connection relationship of multiple first detection subnetworks, input the edge enhancement feature map and the first input data of the first edge enhancement subnetwork corresponding to the target detection subnetwork into the target detection subnetwork, and process the edge enhancement feature map and the corresponding first input data of the first edge enhancement subnetwork corresponding to the target detection subnetwork through the target detection subnetwork to obtain the first segmentation map corresponding to the target detection subnetwork.

[0105] Wherein, when the target detection subnetwork is the first first detection subnetwork, its corresponding first input data is the edge enhancement feature map of the first edge enhancement subnetwork specified in the multiple first edge enhancement subnetworks. Exemplarily, when the target detection subnetwork is the first first detection subnetwork, the aforementioned specified first edge enhancement subnetwork can be the first edge enhancement subnetwork corresponding to the second feature map corresponding to the last second extraction layer. When the target detection subnetwork is not the first first detection subnetwork, its corresponding first input data is the output of the previous first detection subnetwork.

[0106] By setting up multiple first detection sub-networks connected in series to mine features in each level (context), that is, using the detection result of the previous first detection sub-network, the current first detection sub-network is guided to obtain the detection result from its corresponding edge enhancement feature map to enhance the detection of the tampered area. The first detection sub-network can be any neural network in the relevant technology that can realize the detection of the tampered area.

[0107] Thereafter, in step S260 , a tampering detection model is trained based on each first segmentation map and the first label data.

[0108] The first label data may include a label segmentation map generated based on the first sample image. At this time, the first electronic device may construct a total loss based on the difference between each first segmentation map and the label segmentation map, and then adjust the model parameters of the tampering detection model to train the tampering detection model with the goal of minimizing the total loss, that is, minimizing the difference between each first segmentation map and the label segmentation map.

[0109] In some other examples, the first label data includes a label segmentation map and a label edge map generated based on the first sample image. Accordingly, in step S260, it may include: training a tampering detection model based on the first tampering edge map, each first segmentation map, and the label tampering edge map and label segmentation map in the first label data. In this example, the first electronic device may jointly determine the total loss based on the difference between the first tampering edge map and the label edge map, and the difference between each first segmentation map and the label segmentation map, and then adjust the model parameters of the tampering detection model to train the tampering detection model with the goal of minimizing the total loss, that is, minimizing the difference between the first tampering edge map and the label edge map, and the difference between each first segmentation map and the label segmentation map.

[0110] In some possible examples, the aforementioned first label data includes a label tampering edge map and a label segmentation map generated based on the first sample image; accordingly, step S260 includes steps 21-24:

[0111] In step 21, a first loss is constructed based on the difference between the first tampered edge map and the label tampered edge map. In some possible examples, the first electronic device can construct a first loss based on the difference between the first tampered edge map and the label tampered edge map based on a preset first loss function for measuring the similarity between two objects, wherein the first loss is negatively correlated with the similarity between the first tampered edge map and the label tampered edge map, and accordingly, the higher the similarity between the first tampered edge map and the label tampered edge map, the smaller the first loss. For example, the first loss function is, for example, a dice loss function.

[0112] In step 22, based on the differences between the label segmentation map and each first segmentation map, a second loss is constructed. In this step, the first electronic device can use a weighted binary cross entropy loss function and / or a weighted IOU (Intersection over Union) loss function to construct sub-losses corresponding to each first segmentation map based on the differences between the label segmentation map and each first segmentation map, and then determine the second loss based on the sub-losses corresponding to each first segmentation map. Exemplarily, the sum of the sub-losses corresponding to each first segmentation map is determined as the second loss; or the average value between the sub-losses corresponding to each first segmentation map is determined as the second loss.

[0113] In step 23, a total loss is constructed, wherein the total loss is positively correlated with the first loss and the second loss. Then, in step 24, the tampering detection model is trained with the goal of minimizing the total loss.

[0114] Specifically, the first electronic device may determine the product of the first loss and its corresponding weight value, and the sum of the product of the second loss and its corresponding weight value as the total loss; then, with the goal of minimizing the total loss, that is, minimizing the difference between the first tampered edge map and the label tampered edge map, and minimizing the difference between the label segmentation map and each first segmentation map, the model parameters of the main branch network, the reversible branch network, the first edge detection network and the first area detection network in the tampering detection model are adjusted to train the tampering detection model.

[0115] The above steps S210-S260 are a model iteration training process for the tamper detection model. In order to train a better tamper detection model, the above process can be iterated multiple times. That is, after step S260, based on the updated model parameters of the tamper detection model, return to step S210.

[0116] The stopping conditions of the above-mentioned model iterative training process may include: the number of iterative training times reaches a preset number threshold, or the iterative training duration reaches a preset duration, or the total loss is less than a set loss threshold, etc.

[0117] In the above-mentioned model iterative training process, card images that do not contain tampered areas can also be used as sample images, and the tampering detection model can be jointly trained with such sample images and their label data (including label edge maps and label segmentation maps, which do not contain information about tampered areas) to improve the accuracy and stability of the detection results of the trained tampering detection model.

[0118] In the above process, setting up a reversible branch network can alleviate the information bottleneck problem caused by the excessive depth of the neural network based on deep learning. That is to say, the depth of the main branch network does not need to be too deep (that is, there is no need for too many first extraction layers in the main branch network), and it can also be combined with the assistance of the reversible branch network to generate reliable gradient information. Based on this type of gradient information, the main branch network can learn features that are more useful for the tampering area detection task, and solve the information bottleneck problem caused by the excessive depth of the neural network based on deep learning.

[0119] Moreover, for the tampering detection model, the feature maps corresponding to the second extraction layers of different layers in the reversible branch network can be used to perform different detection tasks, for example, tampering regions of different sizes can be detected respectively, wherein the second extraction layers with smaller layers can be guided to learn the features of smaller tampering regions in the image, and the second extraction layers with larger layers can be guided to learn the features of larger tampering regions in the image. Furthermore, when constructing the total loss, the first segmentation maps corresponding to the multiple second extraction layers output by the first region detection network are combined, so that the total loss aggregates the information of the tampering regions of the multiple layers of the second extraction layers in the reversible branch network, and then based on the gradient information obtained by the total loss, the gradient information of the tampering regions corresponding to the multiple layers of the second extraction layers in the reversible branch network is aggregated, and then reversely transmitted to the main branch network, so that the main branch network can learn the features of tampering regions of various sizes, and can retain more local detail information, so that the tampering detection model can realize tampering regions of different sizes, and improve the accuracy of the tampering region detection results of the tampering detection model.

[0120] In some other possible examples, the first label data may further include a label value for indicating whether the first sample image contains a tampered area; Figure 3D As shown, the tampering detection model includes a main branch network, a reversible branch network, a first edge detection network, a first area detection network and a classification network, wherein the main branch network includes multiple first extraction layers connected in series, and the reversible branch network includes multiple second extraction layers connected in series.

[0121] In some possible examples, Figure 3DBased on the structure shown, in the training process of the tampering detection model, after the first electronic device obtains each first segmentation map through the aforementioned steps S210-S250, in step S260, the following steps 31-32 may be included: In step 31, using the classification network, based on the target segmentation image in each first segmentation image, a prediction value for indicating whether the first sample image contains a tampered area is determined. The target segmentation image may be all the first segmentation images, or the segmentation map output by the last first detection subnetwork in each first segmentation image.

[0122] Exemplarily, the classification network may include at least a classifier (eg, a multilayer perceptron or other classifier) ​​to determine, through the classifier, a prediction value indicating whether the first sample image contains a tampered area based on the target segmented image.

[0123] Then, in step 32, the tampering detection model is trained based on the difference between the label value and the predicted value. In this step, a total loss is constructed based on the difference between the label value and the predicted value, and the model parameters of the tampering detection model are adjusted to train the tampering detection model with the goal of minimizing the total loss, that is, minimizing the difference between the label value and the predicted value.

[0124] In some possible examples, Figure 3D Based on the structure shown, in the training process of the tampering detection model, after the first electronic device obtains each first segmentation image and the first tampering edge image through the aforementioned steps S210-S250, in step S260, the following steps 41-42 may be included: In step 41, a classification network is used to determine a prediction value indicating whether the first sample image contains a tampered area based on the target segmentation image and the first tampering edge image in each first segmentation image. Then, in step 42, the tampering detection model is trained based on the difference between the label value and the prediction value.

[0125] In one implementation, the classification network may include two feature extraction layers a arranged in parallel, a feature fusion layer connected to the two feature extraction layers a, and a classifier connected to the feature fusion layer. Accordingly, the first electronic device may input the target segmentation image into a feature extraction layer a1; and input the first tampering edge map into another feature extraction layer a2; then the outputs of the two feature extraction layers are input into the feature fusion layer to obtain the fused feature map; and the fused feature map is processed by the classifier to obtain the predicted value. Then, based on the difference between the label value and the predicted value, the tampering detection model is trained. The feature extraction layer may be implemented based on any network capable of feature extraction in the related art. In some implementations, the feature extraction layer may include a GeM pooling layer (Generalized Mean pooling layer) to extract more features that are beneficial to the tampering area detection task from the target segmentation image and the first tampering edge map, thereby improving the accuracy of the predicted value. The feature fusion layer may achieve the fusion of the two inputs by adding the two inputs pixel by pixel, or multiplying or splicing the two inputs pixel by pixel.

[0126] In another implementation, the classification network may include a first classification subnetwork and a second classification subnetwork, the first electronic device may process the target segmentation image through the first classification subnetwork to obtain a first value, process the first tampering edge map through the second classification subnetwork to obtain a second value, perform weighted summation on the first value and the second value to obtain a predicted value, and then train a tampering detection model based on the difference between the label value and the predicted value. Each classification subnetwork may include at least a classifier; a feature extraction layer may also be included before the classifier.

[0127] In another implementation, the classification network can also be a third classification sub-network, and the first electronic device can also process the feature map of the last second extraction layer in the plurality of second feature maps through the third classification sub-network to obtain a third value; then, the first value, the second value and the third value are weighted and summed to obtain a predicted value, and then the tampering detection model is trained based on the difference between the label value and the predicted value. This is to achieve a joint decision-making mechanism that comprehensively considers multiple information channels, so as to better improve the tampering detection accuracy of the tampering detection model. The structure of the third classification sub-network can refer to the structure of the first classification sub-network and the second classification sub-network.

[0128] In another implementation, after the first electronic device obtains each first segmentation map, each second segmentation map, the first tampered edge map, the second tampered edge map, multiple second feature maps and multiple first feature maps, the aforementioned step 41 may also include: determining a prediction value based on each first segmentation map, each second segmentation map, the first tampered edge map, the second tampered edge map, multiple second feature maps and multiple first feature maps through a classification network.

[0129] In another implementation, after training the main branch network, reversible branch network, first edge detection network and first area detection network in the tampering detection model, the classification network can be trained based on the aforementioned sample images and their label data. When training the classification network, the model parameters of the main branch network, reversible branch network, first edge detection network and first area detection network can be fixed.

[0130] Figure 4 A flow chart of a method for training a tamper detection model in another embodiment of the present specification is shown. The method is executed by a first electronic device, and the first electronic device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. The tamper detection model includes a main branch network, a reversible branch network, a first edge detection network, a first area detection network, a second edge detection network, and a second area detection network. The main branch network includes a plurality of first extraction layers connected in series, and the reversible branch network includes a plurality of second extraction layers connected in series. During the training process of the tamper detection model, as Figure 4 As shown, the method includes the following steps S410-S480:

[0131] In step S410, a first sample image and its first label data are obtained, wherein the first sample image is a card image including a tampered area.

[0132] In step S420, first feature extraction is performed on the first sample image based on the main branch network to obtain multiple first feature maps corresponding to multiple specified first extraction layers.

[0133] In step S430, the first sample image is processed based on the reversible branch network to obtain multiple second feature maps corresponding to multiple specified second extraction layers, wherein the processing includes, through a third extraction layer among the multiple second extraction layers, performing second feature extraction on the second feature map corresponding to the previous layer and the first feature map set corresponding to it to obtain its output feature map, and obtaining the corresponding second feature map based on the output feature map and the first feature map of the first extraction layer of the same layer.

[0134] In step S440, a first tampering edge map is obtained based on the plurality of second feature maps through a first edge detection network.

[0135] Among them, the implementation principle of steps S410-S440 is the same as Figure 2 The implementation principles of steps S210-S240 are similar, and the implementation process can refer to the implementation process of steps S210-S240, which will not be repeated here.

[0136] In step S450, a second tampering edge map is obtained based on the plurality of first feature maps through a second edge detection network. Figure 5 As shown, the structure of the second edge detection network is similar to that of the first edge detection network, and its specific structure can refer to the specific structure of the first edge detection network. The implementation principle of step S450 is similar to that of the aforementioned step S440, and its implementation process can refer to the implementation process of step S440, which will not be repeated here.

[0137] In step S460, a plurality of first segmentation maps are obtained based on the plurality of second feature maps and the first tampering edge map through the first region detection network, wherein the first segmentation map includes the predicted tampering region in the first sample image. The implementation principle of step S460 is similar to Figure 2 The implementation principle of step S250 is similar, and its implementation process can refer to the implementation process of step S250, which will not be repeated here.

[0138] In step S470, a second region detection network is used to obtain a plurality of second segmentation maps based on the second tampering edge map and the plurality of first feature maps. The structure of the second region detection network is similar to that of the first region detection network, such as Figure 5 As shown, the second edge detection network includes a plurality of second detection sub-networks connected in series (including Figure 5 , "CAM4", "CAM5" and "CAM6" as shown), and the second edge enhancement sub-networks corresponding to the specified first extraction layers (including Figure 5 As shown, "EFM5" corresponding to extraction layer 12 is input as the first feature map f12 corresponding to extraction layer 12; "EFM6" corresponding to extraction layer 13 is input as the first feature map f13 corresponding to extraction layer 13; "EFM7" corresponding to extraction layer 14 is input as the first feature map f14 corresponding to extraction layer 14; "EFM8" corresponding to extraction layer 15 is input as the first feature map f15 corresponding to extraction layer 15), wherein the structures of the second detection subnetwork and the second edge enhancement subnetwork are similar to the structures of the first detection subnetwork and the first edge enhancement subnetwork in the first region detection network, respectively, and their specific structures can refer to the specific structures of the first detection subnetwork and the first edge enhancement subnetwork, respectively. The implementation principle of step S470 is similar to that of the aforementioned step S460, and its implementation process can refer to the implementation process of step S460, which will not be repeated here.

[0139] In step S480, a tampering detection model is trained based on each first segmentation map, each second segmentation map and the first label data.

[0140] Exemplarily, the first label data includes a label segmentation map generated based on the first sample image. Accordingly, the first electronic device can construct sub-losses corresponding to each segmentation map (including the first segmentation map and the second segmentation map) based on the label segmentation map and the differences between each first segmentation map and each second segmentation map; then, the sub-losses are combined to determine the total loss, and the total loss is positively correlated with each sub-loss; then, the tampering detection model is trained with the goal of minimizing the total loss, that is, minimizing each sub-loss.

[0141] In another exemplary embodiment, the first label data includes a label edge map and a label segmentation map generated based on the first sample image. Accordingly, in step S480, the first electronic device can construct a first loss based on the difference between the first tampered edge map and the label tampered edge map; construct a second loss based on the difference between the label segmentation map and each first segmentation map; construct a third loss based on the difference between the second tampered edge map and the label tampered edge map; construct a fourth loss based on the difference between the label segmentation map and each second segmentation map; combine the first loss, the second loss, the third loss and the fourth loss to construct a total loss, wherein the total loss is positively correlated with the first loss, the second loss, the third loss and the fourth loss respectively; then, with the goal of minimizing the total loss, adjust the model parameters of the main branch network, the reversible branch network, the first edge detection network, the first region detection network, the second edge detection network and the second region detection network in the tampering detection model to train the tampering detection model.

[0142] The specific process of determining the third loss and the fourth loss may refer to the specific process of determining the first loss and the second loss mentioned above, which will not be elaborated here.

[0143] In the above process, the feature maps corresponding to the second extraction layers of different layers in the reversible branch network of the tampering detection model can be used to perform different detection tasks, and the feature maps corresponding to the first extraction layers of different layers in the main branch network can be used to perform different detection tasks, for example, tampering areas of different sizes can be detected respectively, wherein the second extraction layer (first extraction layer) with a smaller number of layers can be guided to learn the features of smaller tampering areas in the image, and the second extraction layer (first extraction layer) with a larger number of layers can be guided to learn the features of larger tampering areas in the image. Subsequently, when constructing the total loss, the first segmentation maps corresponding to the multiple second extraction layers output by the first region detection network and the second segmentation maps corresponding to the multiple first extraction layers output by the second region detection network are combined, so that the total loss aggregates the information of the tampered areas of the multiple layers of second extraction layers in the reversible branch network and the information of the tampered areas of the multiple layers of first extraction layers in the main branch network, and then based on the gradient information obtained by the total loss, the gradient information of the tampered areas corresponding to the multiple layers of second extraction layers in the reversible branch network and the gradient information of the tampered areas of the multiple layers of first extraction layers in the main branch network are aggregated, and then transmitted back to the main branch network, so that the main branch network does not need to set more first extraction layers, and can still better learn the characteristics of tampered areas of various sizes, and can retain more local detail information, so that the tampering detection model can realize tampering areas of different sizes, better improve the accuracy of the tampering detection result of the tampering detection model, and solve the information bottleneck problem caused by the excessive depth of the neural network based on deep learning.

[0144] Figure 6 The flowchart of a tamper detection method in one embodiment of the present specification is shown. The method is executed by a second electronic device, which can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. The second electronic device can be the same physical device as the first electronic device, or a different physical device. In the tamper detection process, if Figure 6 As shown, the method includes the following steps S610-S660:

[0145] In step S610, a training method based on the aforementioned tampering detection model, a main branch network, a first edge detection network, and a first regional detection network in the trained tampering detection model are obtained. In this step, the second electronic device can obtain the main branch network, the first edge detection network, and the first regional detection network in the tampering detection model from the designated storage area. The training process of the tampering detection model is described in the above embodiment and will not be repeated here. It can be understood that the first electronic device can store the trained tampering detection model in the aforementioned designated storage area.

[0146] In step S620, an image to be detected containing the card to be detected is obtained. In this step, the image to be detected may be an image containing the card to be detected uploaded by a user, or an image sent by other electronic devices that needs to be tampered with by the card to be detected. The card to be detected may include but is not limited to a bank card, an identity card, etc.

[0147] Afterwards, in step S630, based on the main branch network, the first feature extraction is performed on the image to be detected to obtain multiple third feature maps corresponding to multiple specified first extraction layers. The implementation principle of step S630 is similar to that of the aforementioned step S220, and its implementation process can refer to the implementation process of the aforementioned step S220, which will not be described in detail here.

[0148] In step S640, a third tampering edge map is obtained based on the plurality of third feature maps through the first edge detection network. The implementation principle of step S640 is similar to that of the aforementioned step S240, and its implementation process can refer to the implementation process of the aforementioned step S240, which will not be described in detail here. Figure 7 As shown, based on the feature maps corresponding to the target first extraction layer in multiple third feature maps (such as Figure 7 The feature map corresponding to the second first extraction layer as shown in FIG2) and the feature map corresponding to the last first extraction layer as shown in FIG3 Figure 7 As shown in the feature diagram fa5), the number of layers of the target first extraction layer in the main branch network is not higher than the preset number of layers.

[0149] In step S650, a third segmentation map is obtained based on the third tampering edge map and the plurality of third feature maps through the first region detection network. The implementation principle of step S650 is similar to that of the aforementioned step S250, and its implementation process can refer to the implementation process of the aforementioned step S250, which will not be described in detail here.

[0150] In some possible examples, the first area detection network includes: a plurality of first detection sub-networks connected in series, and the third segmentation map is a segmentation map output by the last first detection sub-network.

[0151] In step S660, a detection result of whether the card to be detected contains a tampered area is determined based at least on the third segmentation map.

[0152] In this step, the second electronic device can directly determine the detection result of whether the card to be detected contains a tampered area based on whether the third segmented image contains an image area with a pixel value of the first value. If the third segmented image contains an image area with a pixel value of the first value, and the number of pixel values ​​of the first value in the image area is not less than a preset number, it can be determined that the detection result is that the card to be detected contains a tampered area; otherwise, it is determined that the detection result is that the card to be detected does not contain a tampered area.

[0153] In some possible examples, the second electronic device may also obtain a classification network of a tamper detection model from a designated storage area, and obtain an output value indicating whether the card to be detected contains a tampered area by using the classification network and the third segmentation map, and determine the detection result based on the output value and a preset threshold. If the output value is greater than the preset threshold, it can be determined that the detection result is that the card to be detected contains a tampered area; otherwise, it is determined that the detection result is that the card to be detected does not contain a tampered area.

[0154] It can be understood that the process of obtaining the output value indicating whether the card to be detected contains a tampered area by using the third segmentation map through the classification network can refer to the implementation process of step 31 in the above embodiment, which will not be repeated here.

[0155] In order to obtain a more accurate detection result of whether the card to be tested contains a tampered area, the second electronic device is based on a multi-channel information joint decision-making mechanism, that is, comprehensively considering the third tampering edge map containing edge features and the third segmentation map containing pixel features, and jointly determining the detection result of whether the card to be tested contains a tampered area.

[0156] In view of the above situation, in some possible examples, step S660 may include the following steps 51-52: in step 51: obtain a classification network in the tampering detection model; and in step 52, use the classification network to determine the detection result based on the third tampering edge map and the third segmentation map. In this example, the second electronic device can use the classification network to jointly determine an output value indicating whether the card to be detected contains a tampered area based on the third tampering edge map and the third segmentation map; and then determine the detection result based on the output value and a preset threshold.

[0157] Among them, the process of using the classification network to jointly determine the output value based on the third tampering edge map and the third segmentation map can be referred to the implementation process of step 41 in the above embodiment, which will not be repeated here.

[0158] In some other possible examples, in order to obtain a more accurate detection result of whether the card to be detected contains a tampered area, the second electronic device may also combine multiple third feature maps containing global features, a third tampered edge map containing edge features, and a third segmentation map containing pixel features to jointly determine the detection result of whether the card to be detected contains a tampered area. Exemplarily, the classification network may include a first classification subnetwork, a second classification subnetwork, and a third classification subnetwork; accordingly, in step S660, the following steps 61-64 may be included:

[0159] In step 61, through the aforementioned third classification sub-network, based on the fourth feature map corresponding to the last extraction layer of the main branch network in the plurality of third feature maps, a first result of whether the card to be detected contains a tampered area is determined.

[0160] In some examples, the second electronic device can determine the fourth feature map corresponding to the last extraction layer of the main branch network from the plurality of third feature maps, and then input the fourth feature map into the third classification subnetwork to obtain a first result indicating whether the card to be detected contains a tampered area. In some possible examples, the first result includes a fourth value indicating that the card to be detected contains a tampered area, wherein the higher the fourth value, the greater the possibility that the card to be detected includes a tampered area.

[0161] Among them, Figure 7 As shown, the third classification sub-network can be set after the main branch network, and its input can be the output corresponding to the last extraction layer of the main branch network.

[0162] In some other examples, the second electronic device may also jointly determine the first result based on the fourth feature map and at least one other feature map in the plurality of third feature maps during the process of determining the aforementioned first result. Specifically, the second electronic device may input the fourth feature map and at least one other feature map in the plurality of third feature maps into the third classification subnetwork, so as to fuse the fourth feature map and at least one other feature map in the plurality of third feature maps through the third classification subnetwork to obtain a fused feature, and then process the fused feature through the third classification subnetwork to obtain a first result indicating whether the card to be detected contains a tampered area.

[0163] In step 62, a second result is determined to include a tampered area in the card to be detected based on the third tampered edge graph through the aforementioned second classification sub-network.

[0164] In some examples, the second electronic device can determine, based on the third tampering edge graph, through the second classification subnetwork, a second result that the card to be detected contains a tampered area. Exemplarily, the second result includes a fifth value indicating whether the card to be detected contains a tampered area, wherein the higher the fifth value, the greater the possibility that the card to be detected includes a tampered area.

[0165] like Figure 7 As shown, the second classification subnetwork can be set after the first edge detection network, and its input is the output corresponding to the first edge detection network (for example, the third tampered edge map).

[0166] In step 63, a third result is determined to include a tampered area in the card to be detected based on the third segmentation graph through the aforementioned first classification sub-network.

[0167] In some examples, the second electronic device can determine, based on the third segmentation graph, through the first classification subnetwork, a third result that the card to be detected contains a tampered area. Exemplarily, the third result includes a sixth value indicating whether the card to be detected contains a tampered area, wherein the higher the sixth value, the greater the possibility that the card to be detected contains a tampered area.

[0168] like Figure 7 As shown, the first classification subnetwork can be set after the first area detection network, and its input is the output corresponding to each first detection subnetwork in the first area detection network, or it can be the output corresponding to the last first detection subnetwork in the first area detection network.

[0169] In step 64, the first result, the second result, and the third result are combined to determine the detection result. In some possible examples, when the first result includes the fourth value, the second result includes the fifth value, and the third result includes the sixth value, the second electronic device can determine a comprehensive value based on the fourth value and its corresponding weight coefficient, the fifth value and its corresponding weight coefficient, and the sixth value and its corresponding weight coefficient; if the comprehensive value is greater than a specified threshold, the detection result indicates that the card to be detected contains a tampered area; otherwise, if the comprehensive value is not greater than the specified threshold, the detection result indicates that the card to be detected does not contain a tampered area.

[0170] In some examples, the specified threshold and the weight coefficients corresponding to the fourth value, the fifth value, and the sixth value can be adjusted according to the specific scenario.

[0171] In some possible examples, when a user is handling a designated business online and needs to use his or her card to verify his or her identity, the user can upload an image containing his or her card to be detected, namely, the image to be detected. After obtaining the image to be detected, the second electronic device needs to use the tampering detection method provided in this embodiment to detect tampering of the card to be detected in the image to be detected, that is, to detect whether it contains a tampered area. If it is determined that the card to be detected does not contain a tampered area, the relevant processing procedures for the subsequent designated business can continue to be executed; if it is determined that the card to be detected contains a tampered area, the second electronic device can send a prompt message and can stop the relevant processing procedures for the subsequent designated business.

[0172] It is understandable that in the process of training the tampering detection model, the reversible branch network is used to assist the main branch network to learn more information in the image and ensure the accuracy of the detection results. Accordingly, in the process of tampering detection, in order to improve the detection efficiency, the main branch network, the first edge detection network and the first area detection network in the tampering detection model can be used to jointly execute the tampering detection process to obtain a detection result with higher accuracy. In addition, in the above process, the edge features in the image to be detected are enhanced based on the first edge detection network and the first area detection network, and a multi-channel information joint decision-making mechanism is introduced to better improve the accuracy of tampering area detection.

[0173] In some possible examples, the method may further include steps 71-73:

[0174] In step 71, the reversible branch network in the tampering detection model is obtained. In this step, the second electronic device can obtain the reversible branch network in the tampering detection model from the aforementioned designated storage area.

[0175] In step 72, the image to be detected is processed based on a reversible branch network to obtain multiple fifth feature maps corresponding to multiple specified second extraction layers, wherein the processing includes, through the third extraction layer among the multiple second extraction layers, performing second feature extraction on the fifth feature map corresponding to the previous layer and the third feature map set corresponding to it, to obtain its generated feature map, and based on the generated feature map and the third feature map of the first extraction layer of the same layer, obtaining its corresponding fifth feature map, wherein the third extraction layer is a specified layer among the multiple second extraction layers starting from the second second extraction layer. The implementation principle of step 42 is similar to that of the aforementioned step S230, and its implementation process can refer to the implementation process of step S230, which will not be described in detail here.

[0176] In step 73, a fourth segmentation map is obtained based on the plurality of fifth feature maps through the first region detection network.

[0177] In some possible examples, the second electronic device obtains a fourth segmentation map based on multiple fifth feature maps and the aforementioned third tampered edge map through the first area detection network. In some other possible examples, the second electronic device may also obtain a fourth tampered edge map based on multiple fifth feature maps through the first edge detection network before step 73; and obtain a fourth segmentation map based on multiple fifth feature maps and the fourth tampered edge map through the first area detection network. Among them, the implementation principle of obtaining the fourth segmentation map in step 73 is similar to the implementation principle of obtaining the first segmentation map in the aforementioned step S250. The implementation process can refer to the implementation process of step S250, which will not be repeated here. The implementation principle of obtaining the fourth tampered edge map is similar to the implementation principle of obtaining the first tampered edge map in the aforementioned step S240. The implementation process of obtaining the fourth tampered edge map can refer to the implementation process of step S240, which will not be repeated here.

[0178] In some possible examples, the fourth segmentation map is a segmentation map output by the last first detection sub-network among a plurality of serially connected first detection sub-networks included in the first area detection network.

[0179] Accordingly, step S660 may include: determining whether the card to be detected contains a detection result of a tampered area based at least on the third segmentation map and the fourth segmentation map.

[0180] In a possible implementation, the second electronic device may determine the corresponding results indicating whether the card to be detected contains a tampered area based on the third segmentation map and the fourth segmentation map respectively, and then determine the detection result by combining the corresponding results.

[0181] In another possible implementation, the second electronic device can determine, through the aforementioned classification network, the corresponding determination values ​​indicating whether the card to be detected contains the tampered area based on the third segmentation map and the fourth segmentation map respectively, combined with the corresponding comprehensive values ​​indicating whether the card to be detected contains the tampered area. If the comprehensive value is greater than the aforementioned specified threshold, it is determined that the detection result indicates that the card to be detected contains the tampered area; otherwise, it is determined that the detection result indicates that the card to be detected does not contain the tampered area.

[0182] In the above implementation, the main branch network, the reversible branch network, the first edge detection network and the first area detection network are jointly determined to determine the detection result, so as to better improve the accuracy of the detection result and improve the accuracy of tampering area detection.

[0183] In another possible implementation, when the second electronic device obtains the fourth tampered edge map based on multiple fifth feature maps through the first edge detection network, the aforementioned detection result of determining whether the card to be detected contains a tampered area may also include: based on the fourth feature map, the feature map of the last first extraction layer in the multiple fifth feature maps, the third tampered edge map, the fourth tampered edge map, the third segmentation map and the fourth segmentation map, jointly determining whether the card to be detected contains a comprehensive value of the tampered area, wherein this process and the aforementioned process of determining the comprehensive value are not repeated here. Then, the detection result is determined based on the comprehensive value and the specified threshold.

[0184] In the above example, the detection result of whether the card to be detected contains a tampered area can be determined by combining information from more angles to better improve the accuracy of tampered area detection.

[0185] The foregoing describes certain embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0186] Corresponding to the above method embodiment, the present specification embodiment provides a training device 800 for a tamper detection model, and its schematic block diagram is as follows: Figure 8 As shown, the tampering detection model includes a main branch network, a reversible branch network, a first edge detection network and a first area detection network, the main branch network includes a plurality of first extraction layers connected in series, the reversible branch network includes a plurality of second extraction layers connected in series, and the device includes:

[0187] A first acquisition module 810 is configured to acquire a first sample image and first label data thereof, wherein the first label data is used to indicate whether the first sample image contains a tampered area;

[0188] A first extraction module 820 is configured to perform first feature extraction on the first sample image based on the main branch network to obtain a plurality of first feature maps corresponding to a plurality of specified first extraction layers;

[0189] The first processing module 830 is configured to process the first sample image based on the reversible branch network to obtain a plurality of second feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes, through a third extraction layer among the plurality of second extraction layers, performing second feature extraction on the second feature map corresponding to the previous layer and the first feature map set corresponding to the third extraction layer to obtain its output feature map, and obtaining the corresponding second feature map based on the output feature map and the first feature map of the first extraction layer at the same layer, wherein the third extraction layer is a specified layer among the plurality of second extraction layers starting from the second second extraction layer;

[0190] A first obtaining module 840 is configured to obtain a first tampering edge map based on a plurality of second feature maps through the first edge detection network;

[0191] A second obtaining module 850 is configured to obtain a plurality of first segmentation maps based on a plurality of second feature maps and the first tampering edge map through the first region detection network;

[0192] The training module 860 is configured to train the tampering detection model based on each first segmentation map and the first label data.

[0193] In some possible examples, the tampering detection model further includes a second edge detection network and a second area detection network, and the apparatus further includes:

[0194] a fifth obtaining module (not shown in the figure), configured to obtain a second tampering edge map based on a plurality of first feature maps through the second edge detection network;

[0195] a sixth obtaining module (not shown in the figure), configured to obtain a plurality of second segmentation maps based on the second tampering edge map and the plurality of first feature maps through the second region detection network;

[0196] The training module 860 is specifically configured to train the tampering detection model based on the first tampering edge map, the second tampering edge map, each first segmentation map, each second segmentation map and the first label data.

[0197] In some possible examples, the first label data includes a label tampering edge map and a label segmentation map generated based on the first sample image; the training module 860 is specifically configured to train the tampering detection model based on the first tampering edge map, each first segmentation map and the label tampering edge map and label segmentation map in the first label data.

[0198] In some possible examples, the training module 860 is specifically configured to construct a first loss based on the difference between the first tampering edge map and the label tampering edge map; construct a second loss based on the difference between the label segmentation map and each first segmentation map; construct a total loss, wherein the total loss is positively correlated with the first loss and the second loss; and train the tampering detection model with the goal of minimizing the total loss.

[0199] In some possible examples, the first label data includes a label value for indicating whether the first sample image includes a tampered area; the tampering detection model also includes a classification network;

[0200] The training module 860 is specifically configured to utilize the classification network to determine a prediction value indicating whether the first sample image contains a tampered area based on the target segmented image in each first segmented image and the first tampered edge map; and train the tampering detection model based on the difference between the label value and the prediction value.

[0201] In some possible examples, the first area detection network includes: a plurality of first detection sub-networks connected in series, and first edge enhancement sub-networks corresponding to respective designated second extraction layers;

[0202] The second obtaining module 850 is specifically configured to process the second feature graph and the first tampered edge graph corresponding to each first edge enhancement sub-network respectively, to obtain the edge enhancement feature graph of each first edge enhancement sub-network;

[0203] By processing the edge enhancement feature map and the corresponding first input data of the corresponding first edge enhancement subnetwork of any target detection subnetwork among the multiple first detection subnetworks, a first segmentation map corresponding to the target detection subnetwork is obtained, wherein, when the target detection subnetwork is the first first detection subnetwork, the corresponding first input data is the edge enhancement feature map of the first edge enhancement subnetwork specified among the multiple first edge enhancement subnetworks; when the target detection subnetwork is not the first first detection subnetwork, the corresponding first input data is the output of the previous first detection subnetwork.

[0204] In some possible examples, the multiple second feature maps include: a feature map corresponding to a target second extraction layer of the reversible branch network and a feature map corresponding to a last second extraction layer, and the number of layers of the target second extraction layer in the reversible branch network is not higher than a preset number of layers;

[0205] The first obtaining module 840 is configured to input the feature map corresponding to the target second extraction layer and the feature map corresponding to the last second extraction layer into the first edge detection network to determine the first tampered edge map.

[0206] In some possible examples, the third extraction layer is the i-th extraction layer in the reversible branch network; the first feature map corresponding to the third extraction layer is the first feature map of the i+n-th first extraction layer in the main branch network, where n is a positive integer.

[0207] Corresponding to the above method embodiment, the present specification embodiment provides a tamper detection device 900, whose schematic block diagram is as follows: Fig. 9 As shown, the device includes: a second acquisition module 910, configured to acquire a training device based on the tampering detection model provided by the aforementioned embodiment, a main branch network, a first edge detection network and a first area detection network in the tampering detection model obtained by training; a third acquisition module 920, configured to acquire an image to be detected containing a card to be detected; a second extraction module 930, configured to perform a first feature extraction on the image to be detected based on the main branch network, and obtain a plurality of third feature maps corresponding to a plurality of specified first extraction layers; a third acquisition module 940, configured to obtain a third tampering edge map based on a plurality of third feature maps through the first edge detection network; a fourth acquisition module 950, configured to obtain a third segmentation map based on the first tampering edge map and a plurality of third feature maps through the first area detection network; a determination module 960, configured to determine whether the card to be detected contains a detection result of a tampering area based at least on the third segmentation map.

[0208] In some possible examples, the determination module 960 is specifically configured to obtain a classification network in the tampering detection model; and determine the detection result based on the third tampering edge map and the third segmentation map by using the classification network.

[0209] In some possible examples, the device further includes:

[0210] A fourth acquisition module (not shown in the figure), configured to acquire a reversible branch network in the tampering detection model;

[0211] A second processing module (not shown in the figure) is configured to process the image to be detected based on the reversible branch network to obtain a plurality of fifth feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes, through a third extraction layer among the plurality of second extraction layers, performing second feature extraction on a fifth feature map corresponding to a previous layer and a third feature map set corresponding to the third extraction layer to obtain a generated feature map thereof, and obtaining a corresponding fifth feature map thereof based on the generated feature map and the third feature map of a first extraction layer of the same layer thereof, wherein the third extraction layer is a specified layer among the plurality of second extraction layers starting from the second second extraction layer;

[0212] a fifth obtaining module (not shown in the figure), configured to obtain a fourth segmentation map based on a plurality of fifth feature maps through the first region detection network;

[0213] The determination module 960 is specifically configured to determine whether the card to be detected contains a detection result of a tampered area based at least on the third segmentation map and the fourth segmentation map.

[0214] In some possible examples, the first area detection network includes: a plurality of first detection sub-networks connected in series, and the third segmentation map is a segmentation map output by the last first detection sub-network.

[0215] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0216] The embodiments of the present specification also provide a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the training method of the tampering detection model or the tampering detection method provided in the present specification.

[0217] An embodiment of the present specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the training method of the tampering detection model or the tampering detection method provided in the present specification is implemented.

[0218] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium and computing device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0219] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0220] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A training method for a tampering detection model, the tampering detection model comprising a main branch network, a reversible branch network, a first edge detection network and a first area detection network, the main branch network comprising a plurality of first extraction layers connected in series, the reversible branch network comprising a plurality of second extraction layers connected in series, the method include: Acquire a first sample image and first label data thereof, wherein the first label data is used to indicate whether the first sample image contains a tampered area; Performing first feature extraction on the first sample image based on the main branch network to obtain a plurality of first feature maps corresponding to a plurality of specified first extraction layers; Processing the first sample image based on the reversible branch network to obtain a plurality of second feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes, through a third extraction layer among the plurality of second extraction layers, performing second feature extraction on the second feature map corresponding to the previous layer and the first feature map set corresponding to the third extraction layer to obtain its output feature map, and obtaining the corresponding second feature map based on the output feature map and the first feature map of the first extraction layer at the same layer, wherein the third extraction layer is a specified layer among the plurality of second extraction layers starting from the second second extraction layer; Obtaining a first tampering edge map based on a plurality of second feature maps through the first edge detection network; Obtaining a plurality of first segmentation maps based on a plurality of second feature maps and the first tampering edge map through the first region detection network; The tampering detection model is trained based on each first segmentation map and the first label data.

2. The method according to claim 1, in, The first label data includes a label tampering edge map and a label segmentation map generated based on the first sample image; The training of the tampering detection model comprises: The tampering detection model is trained based on the first tampering edge map, each first segmentation map, and the label tampering edge map and the label segmentation map in the first label data.

3. The method according to claim 2, in, The training of the tampering detection model based on the first tampering edge map, each first segmentation map, and the label tampering edge map and the label segmentation map in the first label data includes: constructing a first loss based on a difference between the first tampering edge map and the label tampering edge map; Constructing a second loss based on the differences between the label segmentation map and each of the first segmentation maps; constructing a total loss, wherein the total loss is positively correlated with the first loss and the second loss; The tamper detection model is trained with the goal of minimizing the total loss.

4. The method according to claim 1, wherein the first label data comprises a label value indicating whether the first sample image contains a tampered area. ; The tamper detection model also includes a classification network; The training of the tampering detection model based on each first segmentation map and the first label data includes: Determining, using the classification network, a prediction value indicating whether the first sample image includes a tampered area based on a target segmented image in each of the first segmented images and the first tampered edge map; The tamper detection model is trained based on the difference between the label value and the predicted value.

5. The method according to claim 1, wherein the tamper detection model further comprises a second edge detection network and a second area detection network, and the method further comprises include: Obtaining a second tampering edge map based on the plurality of first feature maps through the second edge detection network; Obtaining a plurality of second segmentation maps based on the second tampering edge map and the plurality of first feature maps through the second region detection network; The training of the tampering detection model comprises: The tampering detection model is trained based on each first segmentation map, each second segmentation map and the first label data.

6. The method according to claim 1, in, The first area detection network includes: a plurality of first detection sub-networks connected in series, and first edge enhancement sub-networks corresponding to the designated second extraction layers respectively; The method of obtaining a plurality of first segmentation maps based on the first tampering edge map and the plurality of second feature maps through the first region detection network includes: Through each first edge enhancement sub-network, the corresponding second feature map and the first tampered edge map are processed respectively to obtain an edge enhancement feature map of each first edge enhancement sub-network; By processing the edge enhancement feature map and the corresponding first input data of the corresponding first edge enhancement subnetwork of any target detection subnetwork among the multiple first detection subnetworks, a first segmentation map corresponding to the target detection subnetwork is obtained, wherein, when the target detection subnetwork is the first first detection subnetwork, the corresponding first input data is the edge enhancement feature map of the first edge enhancement subnetwork specified among the multiple first edge enhancement subnetworks; when the target detection subnetwork is not the first first detection subnetwork, the corresponding first input data is the output of the previous first detection subnetwork.

7. The method according to claim 1, in, The multiple second feature maps include: a feature map corresponding to a target second extraction layer of the reversible branch network and a feature map corresponding to a last second extraction layer, wherein the number of layers of the target second extraction layer in the reversible branch network is not higher than a preset number of layers; The determining, by the first edge detection network, a first tampering edge graph based on a plurality of second feature graphs includes: The feature map corresponding to the target second extraction layer and the feature map corresponding to the last second extraction layer are input into the first edge detection network to determine the first tampering edge map.

8. The method according to claim 1, in, The third extraction layer is the i-th extraction layer in the reversible branch network; the first feature map corresponding to the third extraction layer is the first feature map of the i+n-th first extraction layer in the main branch network, where n is a positive integer.

9. A tampering detection method, include: Acquire a training method for a tampering detection model according to any one of claims 1 to 8, and a main branch network, a first edge detection network, and a first area detection network in the tampering detection model obtained by training; Acquire the image to be detected containing the card to be detected; Based on the main branch network, performing first feature extraction on the image to be detected to obtain a plurality of third feature maps corresponding to a plurality of specified first extraction layers; Obtaining a third tampering edge map based on a plurality of third feature maps through the first edge detection network; Obtaining, by the first region detection network, a third segmentation map based on the third tampering edge map and a plurality of third feature maps; A detection result of whether the card to be detected contains a tampered area is determined based at least on the third segmentation map.

10. The method according to claim 9, in, The step of determining whether the card to be detected contains a detection result of a tampered area includes: Obtaining a classification network in the tampering detection model; The detection result is determined by using the classification network based on the third tampering edge map and the third segmentation map.

11. The method according to claim 9, in, The method further comprises: Obtaining a reversible branch network in the tamper detection model; Processing the image to be detected based on the reversible branch network to obtain a plurality of fifth feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes, through a third extraction layer among the plurality of second extraction layers, performing second feature extraction on a fifth feature map corresponding to a previous layer and a third feature map set corresponding to the third extraction layer to obtain a generated feature map thereof, and obtaining a corresponding fifth feature map thereof based on the generated feature map and the third feature map of a first extraction layer of the same layer thereof, wherein the third extraction layer is a specified layer among the plurality of second extraction layers starting from the second second extraction layer; Obtaining a fourth segmentation map based on the plurality of fifth feature maps through the first region detection network; The step of determining whether the card to be detected contains a detection result of a tampered area includes: A detection result of whether the card to be detected contains a tampered area is determined based at least on the third segmentation map and the fourth segmentation map.

12. The method according to claim 9, in, The first area detection network includes: a plurality of first detection sub-networks connected in series, and the third segmentation map is a segmentation map output by the last first detection sub-network.

13. A training device for a tampering detection model, the tampering detection model comprising a main branch network, a reversible branch network, a first edge detection network and a first area detection network, the main branch network comprising a plurality of first extraction layers connected in series, the reversible branch network comprising a plurality of second extraction layers connected in series, the device include: A first acquisition module, configured to acquire a first sample image and first label data thereof, wherein the first label data is used to indicate whether the first sample image contains a tampered area; A first extraction module is configured to perform first feature extraction on the first sample image based on the main branch network to obtain a plurality of first feature maps corresponding to a plurality of specified first extraction layers; A first processing module is configured to process the first sample image based on the reversible branch network to obtain a plurality of second feature maps corresponding to a plurality of specified second extraction layers, wherein the processing includes, through a third extraction layer among the plurality of second extraction layers, performing second feature extraction on a second feature map corresponding to a previous layer and a first feature map set corresponding to the third extraction layer to obtain an output feature map thereof, and obtaining a corresponding second feature map based on the output feature map and a first feature map of a first extraction layer at the same layer thereof, wherein the third extraction layer is a specified layer among the plurality of second extraction layers starting from the second second extraction layer; A first obtaining module is configured to obtain a first tampering edge map based on a plurality of second feature maps through the first edge detection network; a second obtaining module configured to obtain a plurality of first segmentation maps based on a plurality of second feature maps and the first tampering edge map through the first region detection network; A training module is configured to train the tampering detection model based on each first segmentation map and the first label data.

14. A tamper detection device, include: A second acquisition module is configured to acquire a training device based on the tampering detection model according to claim 13, a main branch network, a first edge detection network, and a first area detection network in the trained tampering detection model; A third acquisition module is configured to acquire an image to be detected containing a card to be detected; A second extraction module is configured to perform first feature extraction on the image to be detected based on the main branch network to obtain a plurality of third feature maps corresponding to a plurality of specified first extraction layers; A third obtaining module is configured to obtain a third tampering edge map based on a plurality of third feature maps through the first edge detection network; a fourth obtaining module configured to obtain a third segmentation map based on the first tampering edge map and a plurality of third feature maps through the first region detection network; The determination module is configured to determine, based at least on the third segmentation graph, whether the card to be detected contains a detection result of a tampered area.

15. A computing device comprising a memory and a processor, in, The memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 8 or 9 to 12 is implemented.

Citation Information

Patent Citations

  • Image detection method and device

    CN113763405A

  • Image tampering detection method based on edge guidance and multi-level search

    CN116342601A

  • Image tampering detection method and system based on double-flow network

    CN117557562A

  • System and Method for Extremely Efficient Image and Pattern Recognition and Artificial Intelligence Platform

    US20180204111A1