A general identification document anomaly detection model training and using method and device

Abstract

The embodiment of the specification discloses a general certificate anomaly detection model training method, comprising: acquiring a sample image, wherein the sample image comprises an abnormal certificate image and a corresponding labeled image; inputting the abnormal certificate image into a pre-trained segmentation network to classify each pixel point, and the classification result comprises normal pixel points, abnormal pixel points and unknown pixel points; taking each classified unknown pixel point as an abnormal pixel point to obtain a segmentation image; estimating a penalty coefficient corresponding to each pixel point in the segmentation image according to the segmentation network; for each pixel point in the segmentation image, calculating the waiver loss of the classification of each pixel point by the segmentation network according to the penalty coefficient; determining the segmentation loss according to the waiver loss; and fine-tuning the segmentation network according to the segmentation loss. Accordingly, the embodiment of the specification also discloses a general certificate anomaly detection model training device.

Description

Technical Field

[0001] The embodiments in this specification relate to image processing technology, and more particularly to a general method and apparatus for training and using a document anomaly detection model. Background Technology

[0002] During online identity verification, user-uploaded documents may be obscured by fingers, pieces of paper, punched holes, or damaged, resulting in non-compliant photos. Therefore, a document quality compliance algorithm is needed to intercept these non-compliant images. Given hundreds of document types and various obstruction scenarios, while some corresponding foreign object detection algorithms have been developed for different types of obstruction, a more universal foreign object detection algorithm is urgently needed.

[0003] Therefore, the embodiments in this specification aim to provide a general solution for detecting document anomalies, thereby intercepting various types of non-compliant documents. Summary of the Invention

[0004] One of the objectives of the embodiments in this specification is to provide a general method for training a document anomaly detection model. This method can accurately detect abnormal pixels in document images, thereby identifying various types of foreign object occlusion and effectively intercepting non-compliant documents due to occlusion.

[0005] In accordance with the above objectives, this specification proposes a general method for training a document anomaly detection model, wherein the document anomaly detection model includes a segmentation network, and the method includes the following steps:

[0006] Obtain sample images, wherein the sample images include abnormal document images and corresponding labeled images, and the labeled images pre-label abnormal pixels;

[0007] The abnormal document image is input into a pre-trained segmentation network to classify each pixel. The classification results include normal pixels, abnormal pixels, and unknown pixels.

[0008] Each unknown pixel identified in the classification is treated as an anomaly pixel, resulting in a segmented image;

[0009] Based on the segmentation network, estimate the penalty coefficient corresponding to each pixel in the segmented image;

[0010] For each pixel in the segmented image, the abandonment loss of the segmentation network in classifying each pixel is calculated based on the penalty coefficient;

[0011] The partition loss is determined based on the aforementioned waiver loss;

[0012] The segmentation network is fine-tuned based on the segmentation loss.

[0013] In some implementations, the segmentation network can employ U-Net, Fully Convolutional Networks (FCN), and Deeply-Supervised Networks (DSN), which are mainstream technologies in the prior art.

[0014] In the embodiments of this specification, the penalty coefficient is used to evaluate the accuracy of the segmentation network in classifying each pixel, especially to evaluate whether the segmentation network has classified pixels that should have been classified as normal or abnormal pixels as unknown pixels. The abandonment loss calculated based on the penalty coefficient can effectively measure the classification ability of the segmentation network. In addition, the segmentation network model used in the embodiments of this specification is pre-trained on a large-scale dataset to improve the generalization of segmentation and the ability to detect foreign objects. It is then fine-tuned through a foreign object detection task, which can enhance the segmentation network's ability to intercept various abnormal occlusion types while saving computational costs.

[0015] Furthermore, in some embodiments, estimating the penalty coefficient corresponding to each pixel in the segmented image based on the segmentation network includes:

[0016] The segmentation network defines an energy function to determine the energy corresponding to each pixel in the segmented image. The energy is used to comprehensively represent the probability that each pixel belongs to different categories. The energy of unknown pixels is higher than that of other pixels.

[0017] For each pixel in the segmented image, the corresponding penalty coefficient is estimated based on its energy.

[0018] Furthermore, in some implementations, the step after estimating the corresponding penalty coefficient based on the energy of each pixel in the segmented image is as follows:

[0019] Set an energy threshold, and calculate the energy loss based on the energy threshold and the energy corresponding to each pixel in the segmented image;

[0020] For each pixel in the segmented image, the abandonment loss of the segmentation network for classifying each pixel is calculated according to the penalty coefficient, wherein the abandonment loss is negatively correlated with the probability of classifying the pixel as an unknown pixel or the probability of classifying it as another pixel;

[0021] The partition loss is calculated based on the energy loss and the abandonment loss;

[0022] The segmentation network is fine-tuned with the goal of minimizing the segmentation loss.

[0023] Furthermore, in some embodiments, setting an energy threshold and calculating the energy loss based on the energy threshold and the energy corresponding to each pixel in the segmented image specifically includes:

[0024] Set a first energy threshold, and calculate a first energy loss based on the first energy threshold and the energy of unknown pixels in the segmented image;

[0025] A second energy threshold is set, and a second energy loss is calculated based on the second energy threshold and the energy of pixels other than unknown pixels in the segmented image. The second energy threshold is lower than the first energy threshold.

[0026] Furthermore, in some embodiments, calculating the abandonment loss of the segmentation network for classifying each pixel based on the penalty coefficient includes:

[0027] The abandonment loss of the segmentation network for classifying unknown pixels is calculated based on the penalty coefficient, and is used as the first abandonment loss.

[0028] The abandonment loss for the segmentation network to classify pixels other than unknown pixels is calculated based on the penalty coefficient and used as the second abandonment loss.

[0029] Furthermore, in some embodiments, the labeled image includes a mask image.

[0030] Furthermore, in some embodiments, after acquiring the sample image, the abnormal document images in the sample image are aligned.

[0031] Furthermore, in some embodiments, aligning the abnormal document images in the sample images specifically includes:

[0032] Corner point location detection is performed on the abnormal document image;

[0033] Based on the detected corner positions, the abnormal document images are aligned using affine transformation.

[0034] This specification also proposes a general method for using a document anomaly detection model, the method comprising the following steps:

[0035] The client captures images of the documents to be inspected.

[0036] The document image is input into a pre-trained document anomaly detection model to detect and label abnormal regions in the document image. The document anomaly detection model is trained using any of the methods described above.

[0037] Another objective of the embodiments of this specification is to provide a universal document anomaly detection model training device, which can accurately detect abnormal pixels in document images, thereby identifying various types of foreign object occlusion and effectively intercepting non-compliant documents due to occlusion.

[0038] In accordance with the above objectives, embodiments of this specification propose a general training device for a document anomaly detection model, comprising:

[0039] The sample acquisition module is used to acquire sample images, wherein the sample images include abnormal document images and corresponding labeled images, and the labeled images are pre-labeled with abnormal pixels;

[0040] The segmentation module is used to input the abnormal document image into a pre-trained segmentation network to classify each pixel. The classification results of each pixel include normal pixels, abnormal pixels, and unknown pixels. The classified unknown pixels are regarded as abnormal pixels to obtain a segmented image.

[0041] The training module is used to estimate the penalty coefficient corresponding to each pixel in the segmented image based on the segmentation network; calculate the abandonment loss of the segmentation network for classifying each pixel based on the penalty coefficient for each pixel in the segmented image; determine the segmentation loss based on the abandonment loss; and fine-tune the segmentation network based on the segmentation loss.

[0042] In some implementations, the segmentation network can employ U-Net, Fully Convolutional Networks (FCN), and Deeply-Supervised Networks (DSN), which are mainstream technologies in the prior art.

[0043] Furthermore, in some embodiments, the training module defines an energy function based on the segmentation network to determine the energy corresponding to each pixel in the segmented image. The energy is used to represent the combined probability that each pixel belongs to different categories, wherein the energy of normal pixels is lower than that of abnormal pixels. For each pixel in the segmented image, a corresponding penalty coefficient is estimated based on its energy, and the penalty coefficient is negatively correlated with the energy.

[0044] Furthermore, in some implementations, the training module estimates the corresponding penalty coefficient based on the energy of each pixel in the segmented image and then sets it as follows:

[0045] The training module sets an energy threshold and calculates the energy loss based on the energy threshold and the energy corresponding to each pixel in the segmented image.

[0046] For each pixel in the segmented image, the abandonment loss of the segmentation network for classifying each pixel is calculated according to the penalty coefficient. The abandonment loss is negatively correlated with the probability of classifying the pixel as an unknown pixel and the probability of classifying it as other pixels.

[0047] The partition loss is calculated based on the energy loss and the abandonment loss;

[0048] The segmentation network is fine-tuned with the goal of minimizing the segmentation loss.

[0049] Furthermore, in some embodiments, the training module sets an energy threshold, and the energy loss is calculated based on the energy threshold and the energy corresponding to each pixel in the segmented image, specifically including:

[0050] The training module sets a first energy threshold and calculates a first energy loss based on the first energy threshold and the energy of unknown pixels in the segmented image; it sets a second energy threshold and calculates a second energy loss based on the second energy threshold and the energy of pixels other than unknown pixels in the segmented image, wherein the second energy threshold is lower than the first energy threshold.

[0051] Furthermore, in some embodiments, the training module calculates the abandonment loss of the segmentation network for classifying unknown pixels based on the penalty coefficient, as a first abandonment loss; and calculates the abandonment loss of the segmentation network for classifying pixels other than unknown pixels based on the penalty coefficient, as a second abandonment loss.

[0052] Furthermore, in some embodiments, the labeled image includes a mask image.

[0053] Furthermore, in some embodiments, after the sample acquisition module acquires the sample image, it further includes aligning the abnormal document images in the sample image.

[0054] Furthermore, in some embodiments, the sample acquisition module performs corner point location detection on the abnormal document image; based on the detected corner point locations, it uses affine transformation to align the abnormal document image.

[0055] This specification also proposes a general device for using a document anomaly detection model, comprising:

[0056] The image acquisition module is used to acquire images of the documents to be inspected.

[0057] The detection module is used to input the document image into a pre-trained document anomaly detection model, detect abnormal regions in the document image and label them, wherein the document anomaly detection model is trained using any of the methods described above.

[0058] The purpose of this specification is also to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the method described in any one of the above-mentioned embodiments.

[0059] The beneficial effects of the general document anomaly detection model training and usage method described in the embodiments of this specification are that the abandonment loss is calculated based on the penalty coefficient used to evaluate the classification accuracy of the segmentation network, thereby effectively measuring the classification ability of the segmentation network; in addition, the segmentation network model used in the embodiments of this specification is pre-trained on a large-scale dataset to improve the generalization of segmentation and foreign object detection ability, and then fine-tuned through foreign object detection tasks. This can enhance the segmentation network's ability to intercept various types of abnormal occlusion while saving computational costs.

[0060] The document anomaly detection model training and usage device described in the embodiments of this specification also has the above-mentioned beneficial effects. Attached Figure Description

[0061] Figure 1 An exemplary schematic diagram of a scenario in one implementation of the general document anomaly detection model training method described in the embodiments of this specification is shown.

[0062] Figure 2 The diagram illustrates, exemplarily, the structure of the segmentation network in one implementation of the general document anomaly detection model training method described in the embodiments of this specification.

[0063] Figure 3 An exemplary flowchart of one implementation of the general document anomaly detection model training method described in the embodiments of this specification is shown.

[0064] Figure 4 The flowchart of the general document anomaly detection model training method described in the embodiments of this specification is illustrated in one specific implementation.

[0065] Figure 5 An exemplary flowchart is shown in one implementation of the general document anomaly detection model described in the embodiments of this specification.

[0066] Figure 6 An exemplary flowchart of the method for using the general document anomaly detection model described in the embodiments of this specification is shown in one specific implementation.

[0067] Figure 7An exemplary schematic diagram of the structure of the general document anomaly detection model training device described in the embodiments of this specification is shown in one implementation.

[0068] Figure 8 An exemplary schematic diagram of the device used in one embodiment of the general document anomaly detection model described in this specification is shown. Detailed Implementation

[0069] The following will provide a more detailed description of the general document anomaly detection model training and usage method and apparatus described in the embodiments of this specification, in conjunction with the accompanying drawings and specific examples. However, this detailed description does not constitute a limitation on the embodiments of this specification.

[0070] Figure 1 An exemplary schematic diagram of a scenario in one implementation of the general document anomaly detection model training method described in the embodiments of this specification is shown.

[0071] like Figure 1 As shown, the image of the document to be detected for foreign object occlusion is captured by the client's mobile device and input into the server's quality compliance algorithm for detection. If a foreign object occlusion area is detected, the document quality is non-compliant, and the server requires the client to retake the photo. The client adjusts the document to be detected and captures the document image again, and inputs it into the server's quality compliance algorithm. The above process is repeated until no foreign object occlusion area is detected, which indicates that the document quality is compliant.

[0072] In some implementations, the segmentation network can employ mainstream technologies such as U-Net, Fully Convolutional Networks (FCN), and Deeply-Supervised Networks (DSN). The segmentation network analyzes the attributes of pixels in the input image, grouping pixels with similar attributes into the same category and pixels with significantly different attributes into different categories. In document foreign object detection tasks, segmentation networks can directly segment abnormal regions of arbitrary shapes in document images, i.e., foreign object occlusion areas. Furthermore, segmentation networks have low data dependency, requiring only a small number of sample images to train and achieve good segmentation results.

[0073] Figure 2 The diagram illustrates, exemplarily, the structure of the segmentation network in one implementation of the general document anomaly detection model training method described in the embodiments of this specification.

[0074] like Figure 2As shown, in some specific embodiments, the U-Net segmentation network is used as the backbone network of the document anomaly detection model in the embodiments of this specification. After the document image to be detected is input into the segmentation network, various sizes of convolution, ReLU, concatenation, pooling, upsampling, and other processing are performed on the image size and number of channels to obtain the corresponding segmented image and output it. The U-Net segmentation network is prior art, so it will not be described in detail here.

[0075] The embodiments in this specification employ the Abstention Learning (PAL) method, which involves adding a new category to be abandoned during network classification, as the abstention class, as shown in the following formula:

[0076]

[0077] Where x is the model input; f(x) is the model classification result; g h (x) is the selection function; h is the abstention threshold. If the value of the selection function is lower than the threshold, the model abandons classification.

[0078] Accordingly, when calculating the abandonment loss of the model, it can be calculated separately for the normal class and the abandonment class, and hyperparameters can be added to the loss of the normal class or the abandonment class to constrain it according to the actual situation. Abandonment learning can comprehensively handle classes that have not appeared in the training dataset, improving the accuracy and generality of the model classification.

[0079] This specification also employs an energy-based model (EBM) in its embodiments. Essentially, it constructs a function E(x) that maps each point x in the input space to a scalar called energy. The energy can be converted into probability density using the Gibbs distribution.

[0080]

[0081] Where x is the input; y is the category; y′ is a category among all categories; T is the temperature parameter; E(x,y) represents the energy of x when the input x is classified as y.

[0082] If we call the denominator of the above equation the partition function, then the energy of any input point x can be defined as:

[0083] E(x) = -Tlog∫ y′ e ―E(x,y′) / T

[0084] Where E(x) is the energy at x.

[0085] By combining the classification model with the energy model, the input x can be mapped to K values ​​to represent the probability that x is classified into K classes, and the probability distribution is as follows:

[0086]

[0087] Among them, f y (x) represents the probability of classifying input x as y; i is the index of the class; and K is the total number of classes.

[0088] In summary, energy can be defined as:

[0089] E(x,y)=―f y (x)

[0090] The energy of input point x can then be expressed as:

[0091]

[0092] EBM has very few limitations because it doesn't directly model a normalized distribution, but only focuses on non-normalized log probabilities (the energy function). Since the energy function doesn't need normalization, it can be modeled by any regression function. Therefore, classification problems can be simplified to nonlinear regression problems, and the network type can be arbitrarily chosen based on the data format, such as graph networks or convolutional networks. Currently, applications of EBM in machine learning include image generation, NLP, and reinforcement learning.

[0093] In one embodiment of this specification, a general method for training a document anomaly detection model is proposed. Figure 3 An exemplary flowchart of one implementation of the general document anomaly detection model training method described in the embodiments of this specification is shown.

[0094] like Figure 3 As shown, in one embodiment, the method specifically includes the following steps:

[0095] 100: Obtain sample images, which include abnormal document images and corresponding labeled images, with abnormal pixels pre-labeled in the labeled images.

[0096] Sample images can be obtained from known large-scale datasets or acquired on demand using devices with image acquisition capabilities, and the acquired sample images are pre-annotated. Abnormal document images can include various types of foreign object occlusion, such as finger occlusion, paper occlusion, document perforation, and document damage. In the annotated images, occluded areas or areas with missing document content are displayed in a way that differs from other normal areas.

[0097] In some embodiments, the labeled image includes a mask image.

[0098] In the mask image, the positions corresponding to normal pixels are marked as 0, and the positions corresponding to abnormal pixels are marked as 1. Mask images are an existing technology that can be used to extract regions of interest from an image for subsequent analysis, or to mask out regions of uninteresting interest to avoid unnecessary computation and improve the overall detection efficiency of the document anomaly detection model.

[0099] In some embodiments, after acquiring the sample image, the abnormal document images in the sample image are further aligned to remove background factors other than the document itself, so as to unify and standardize the input of the document anomaly detection model.

[0100] In some more specific embodiments, aligning abnormal document images in the sample images specifically includes:

[0101] Corner point location detection for abnormal document images;

[0102] Based on the detected corner positions, affine transformations are used to align the abnormal document images.

[0103] Optionally, corner detection can employ a regression-based method, which uses an existing model to regress and predict the positions of the four corners on the document. Based on the positions of the four corners, affine transformation is used to correct and align the sample image, further standardizing the input of the document anomaly detection model and improving the model's computational efficiency.

[0104] 102: Input the abnormal document image into the pre-trained segmentation network to classify each pixel. The classification results include normal pixels, abnormal pixels, and unknown pixels.

[0105] For existing segmentation network models, the embodiments in this specification employ a combination of pre-training and fine-tuning to train the model. Pre-training the segmentation network model on a large-scale dataset improves the generalization ability of segmentation and foreign object detection. Fine-tuning through a foreign object detection task enhances the segmentation network's ability to intercept various types of abnormal occlusions while saving computational costs.

[0106] As the types of documents differ, the layout of various identity information in the document images also varies. Faced with many types of documents, there are also many types of foreign object occlusion. Different types of foreign object occlusion correspond to abnormal regions with different shapes and sizes. Therefore, the training samples cannot include all documents and all corresponding foreign object occlusion types. If the segmentation network encounters pixels that are difficult to classify, it will directly add a new category, that is, abandon the classification in the existing category and classify the pixels of this category as unknown pixels.

[0107] 104: The unknown pixels identified in the classification are treated as outlier pixels to obtain the segmented image.

[0108] For foreign object occlusion types with known shape and size features, the segmentation network can detect the abnormal region and directly classify the pixels within the region as abnormal pixels. However, for foreign object occlusion types that have not appeared in the training samples, the segmentation network will encounter difficulties in classification. In order to make the segmentation results more accurate, the network will abandon the classification of such pixels in the existing category, assign a new category to the pixels, and then treat them as abnormal pixels.

[0109] By introducing the pixel-level abstention learning (PAL) steps into the segmentation network, the document anomaly detection model can detect more types of foreign object occlusion, including known foreign object occlusion types contained in the training samples and unknown foreign object occlusion types. The detection accuracy for unknown foreign object occlusion types is improved. At the same time, it can reduce the risk of classification errors in the document anomaly detection model.

[0110] 106: Based on the segmentation network, estimate the penalty coefficient corresponding to each pixel in the segmented image.

[0111] The penalty coefficient is used to evaluate the accuracy of the segmentation network in classifying each pixel, especially to evaluate whether the segmentation network has classified pixels that should have been classified as normal or abnormal pixels as unknown pixels. The "penalty" represents the penalty for abandoning the classification of a pixel and assigning it to an unknown pixel; therefore, the penalty coefficient for normal or abnormal pixels is higher than that for unknown pixels, meaning the penalty coefficient for pixels other than unknown pixels is higher than that for unknown pixels.

[0112] In some embodiments, estimating the penalty coefficient corresponding to each pixel in the segmented image based on the segmentation network includes:

[0113] The segmentation network defines an energy function to determine the energy of each pixel in the segmented image. The energy is used to comprehensively represent the probability that each pixel belongs to different categories. The energy of unknown pixels is higher than that of other pixels.

[0114] For each pixel in the segmented image, the corresponding penalty coefficient is estimated based on its energy.

[0115] By establishing an Energy-based Model (EBM), an energy function based on a segmentation network is constructed, combining the segmentation network model with the energy model. Each pixel in the document image is assigned a scalar called energy. The energy of each pixel is obtained by exponentially summing the probabilities of that pixel belonging to different categories. The energy of each pixel is negatively correlated with this exponential sum, and the energy is a negative number. Since the probability of an unknown pixel belonging to a newly added unknown pixel category is much greater than its probability of belonging to other existing categories, and the probability of normal or abnormal pixels belonging to existing categories is much greater than their probability of belonging to unknown pixels, the energy of unknown pixels is higher.

[0116] In some more specific embodiments, for each pixel in the segmented image, a penalty coefficient is calculated based on the square of its energy. Since energy is a negative number, the penalty coefficient for each pixel is negatively correlated with its energy. The energy of each pixel is calculated using the EBM model, and then the penalty coefficient for each pixel is calculated. Normal or abnormal pixels have lower energy and higher penalty coefficients, indicating that they should be classified into existing categories; unknown pixels have higher energy and lower penalty coefficients, indicating that they should not be classified into existing categories.

[0117] The initial PAL model normalized the classification of unknown objects using a predefined fixed penalty coefficient for all pixels. However, in complex scenes, different penalties are often required for different pixels; for example, pixels of small (distant) objects and pixels of large (near) objects require different penalty coefficients. By combining PAL with EBM, an adaptive energy-based dynamic penalty coefficient can be estimated. This penalty estimation can adaptively learn an energy-based penalty to adapt to the diverse pixel features in the scene, thereby achieving accurate detection of different types of foreign objects.

[0118] 108: For each pixel in the segmented image, calculate the abandonment loss of the segmentation network in classifying each pixel based on the penalty coefficient.

[0119] 110: Determine the partition loss based on the loss of waiver.

[0120] 112: Fine-tune the segmentation network based on the segmentation loss.

[0121] In some more specific embodiments, after estimating the corresponding penalty coefficient based on the energy of each pixel in the segmented image, the following steps are used instead of steps 108-112:

[0122] Set an energy threshold, and calculate the energy loss based on the energy threshold and the energy corresponding to each pixel in the segmented image.

[0123] For each pixel in the segmented image, the abandonment loss of the segmentation network for classifying each pixel is calculated based on the penalty coefficient. The abandonment loss is negatively correlated with the probability of classifying the pixel as an unknown pixel or as another pixel.

[0124] The partition loss is calculated based on energy loss and abandonment loss;

[0125] The segmentation network is fine-tuned with the goal of minimizing the segmentation loss.

[0126] The energy threshold is used to compare the energy of each pixel. By minimizing energy loss, the energy of each pixel can be controlled within a specific range during training, so that pixels of different categories have different energy ranges.

[0127] In some more specific embodiments, the penalty coefficient exists as the denominator in the abandonment loss, while the numerator represents the probability of classifying a pixel as an unknown pixel. The probability of classifying a pixel as an existing category is added to the former in a one-term form. Under the training objective of minimizing the abandonment loss, the penalty coefficient is larger for normal or abnormal pixels, so maximizing the probability of classifying a pixel as an existing category is more efficient as the training objective; for unknown pixels, the penalty coefficient is smaller, so maximizing the probability of classifying a pixel as an unknown pixel, which is the numerator, is more efficient.

[0128] The corresponding segmentation loss is calculated by weighted summation of energy loss and abandonment loss, and the entire segmentation network is fine-tuned based on this segmentation loss.

[0129] In some more specific embodiments, setting an energy threshold and calculating the energy loss based on the energy threshold and the energy corresponding to each pixel in the segmented image specifically includes:

[0130] Set a first energy threshold, and calculate a first energy loss based on the first energy threshold and the energy of unknown pixels in the segmented image;

[0131] A second energy threshold is set, and a second energy loss is calculated based on the second energy threshold and the energy of pixels other than unknown pixels in the segmented image. The second energy threshold is lower than the first energy threshold.

[0132] The first energy threshold is used to set a lower limit for the energy of unknown pixels, and the second energy threshold is used to set an upper limit for the energy of normal or abnormal pixels. During training, the gap between the pixel and the corresponding energy threshold is narrowed by minimizing the first and second energy losses, thereby widening the energy gap between normal or abnormal pixels and unknown pixels. Through training, the energy of the pixel can more accurately represent the category of the pixel.

[0133] In some embodiments, calculating the abandonment loss for the segmentation network to classify each pixel based on the penalty coefficient includes:

[0134] The abandonment loss for classifying unknown pixels by the segmentation network is calculated based on the penalty coefficient and used as the first abandonment loss.

[0135] The abandonment loss for classifying pixels other than unknown pixels by the segmentation network is calculated based on the penalty coefficient, and is used as the second abandonment loss.

[0136] The abandonment loss is used to evaluate the accuracy of the segmentation network in classifying each pixel. By using adaptive penalty parameters, the probability of classifying a pixel into a specific category that needs to be maximized can be dynamically selected, thereby accurately detecting different types of abnormal objects.

[0137] In some embodiments, the segmentation network is fine-tuned with the goal of minimizing the segmentation loss.

[0138] To provide a more systematic and complete explanation of the general document anomaly detection model training method in the embodiments of this specification, the following embodiments will be explained in conjunction with illustrations and formulas.

[0139] Figure 4 The flowchart of the general document anomaly detection model training method described in the embodiments of this specification is illustrated in one specific implementation.

[0140] In some specific embodiments, a normal image x is first input to the segmentation network. in and the corresponding labeled image y in Output the probability (Logits) of each existing category for pixels that can be correctly classified, expressed as f. θ (y in ;x in This indicates that the anomalous image x is input into the segmentation network for the second time. out and the corresponding labeled image y out Output the probability that an unknown pixel with an anomaly belongs to each category (including a new category indicating abandonment of classification to an existing category), denoted by f. θ (y out ;x out )express.

[0141] The energy function is defined based on the segmentation network, such that the energy E of each pixel is... θ (x) ω It can be expressed as follows:

[0142]

[0143] Where x represents the input image; y represents the labeled image corresponding to the input image; Y represents the classifiable category; fθ (y;x) ω θ represents the probability that the pixel with index ω in image x is classified as class y; θ is the model parameter; ω represents the index of the pixel.

[0144] The energy E of each pixel in the segmented image of the normal image is calculated according to the above formula. θ (x in The energy E of each pixel in the segmented image of the abnormal image. θ (x out To increase the energy difference between unknown pixels and other pixels, a first energy threshold m is preset. out Second energy threshold m in , where m out Greater than m in .

[0145] The first energy loss is calculated according to the following formula. Second energy loss

[0146]

[0147]

[0148] Among them, l ebm Indicates energy loss; E θ (x) ω This represents the energy corresponding to the pixel with index ω in image x; m out The first energy threshold; m in θ represents the second energy threshold; θ is the model parameter; Ω represents the total number of pixel indices.

[0149] Based on the energy E of each pixel θ (x) ω The penalty parameter 'a' for each pixel can be calculated using the following formula. ω :

[0150] a ω =(―E) θ (x) ω ) 2

[0151] Then the loss for waiving is l pal It can be obtained through the following formula:

[0152]

[0153] Among them, f θ (y ω ;x) ω a represents the probability that a pixel in image x is classified into class y; ωFor the penalty parameter; f θ (Y+1;x) ω This represents the probability that a pixel in image x is classified as class Y+1, which is the probability of it being classified as an unknown pixel.

[0154] Based on the above formula, calculate the abandonment loss of the segmentation network for classifying unknown pixels and other pixels respectively, and obtain the first abandonment loss l. pal (θ,y out ,x,E θ (x) and the second abandonment loss l pal (θ,y in ,x,E θ (x)).

[0155] In some more specific embodiments, a regularization loss is introduced to improve the smoothness and sparsity of the segmentation results. reg :

[0156]

[0157] Where β1 and β2 are regularization parameters; This serves as the index for adjacent pixels. β1 controls the energy difference between adjacent pixels, ensuring the difference isn't too large, thus achieving a smoothing effect. β2 enhances the energy of unknown pixels, widening the gap between them and nearby pixels, thereby highlighting abnormal areas and achieving a sparsity effect.

[0158] The segmentation loss l of the segmentation network is obtained by the following formula:

[0159]

[0160] in, This represents the input normal image and its corresponding labeled image; This represents the input abnormal image and its corresponding labeled image; λ is the model parameter.

[0161] If the segmentation network is fine-tuned with the goal of minimizing the segmentation loss, then the abandonment loss, energy loss, and regularization loss all need to be minimized.

[0162] The general document anomaly detection model training method provided in the embodiments of this specification jointly applies abandonment learning (PAL) and energy-based model (EBM) in the segmentation network. First, image-level abandonment learning is upgraded to pixel-level abandonment learning. By abandoning the classification of unknown pixels into existing categories, various unknown foreign object occlusion types can be effectively identified, ensuring the versatility of the document anomaly detection model. The introduction of pixel energy provides an adaptive penalty coefficient, allowing the document anomaly detection model to dynamically estimate the corresponding penalty based on the energy of specific pixels, thereby achieving more accurate pixel classification and anomaly detection.

[0163] In one embodiment of this specification, a general method for using a document anomaly detection model is also proposed. Figure 5 An exemplary flowchart is shown in one implementation of the general document anomaly detection model described in the embodiments of this specification.

[0164] like Figure 5 As shown, in one embodiment, the method includes:

[0165] 200: The image of the document to be inspected is captured via the client.

[0166] 202: Input the document image into the pre-trained document anomaly detection model, detect abnormal regions in the document image and label them. The document anomaly detection model is trained using the general document anomaly detection model training method described in the embodiments of this specification.

[0167] The client can be a smartphone, smartwatch, camera, or other device with image acquisition capabilities. The client and server can directly transmit images over the network, or the client can use a sub-device connected to the host of the server via a cable to acquire images, such as a camera.

[0168] In some embodiments, after acquiring the image of the document to be detected, the document image is further aligned to remove background factors other than the document itself, thereby unifying and standardizing the input of the document anomaly detection model.

[0169] In some more specific embodiments, aligning the document images specifically includes:

[0170] Corner point detection in document images;

[0171] Based on the detected corner positions, the document image is aligned using affine transformation.

[0172] Optionally, corner detection can employ a regression-based method, which trains an existing model to regress and predict the positions of the four corners on the document. Based on the positions of the four corners, affine transformation is used to correct and align the document image, further standardizing the input of the document anomaly detection model and improving the model's computational efficiency.

[0173] The document image is input into a pre-trained document anomaly detection model. The model determines whether a pixel is normal or abnormal by calculating the energy of each pixel. During model training, by minimizing energy loss, abandonment loss, and regularization loss, the energy difference between normal and abnormal pixels is widened, and the energy of normal pixels is significantly lower than that of abnormal pixels. Therefore, abnormal pixels can be detected based on the energy level.

[0174] In some embodiments, the method for detecting and labeling abnormal regions in an ID card image includes labeling using a mask. In the output mask image, which is the same size as the original ID card image, the positions corresponding to normal pixels are labeled as 0, and the positions corresponding to abnormal pixels are labeled as 1. Mask images are existing technology and can be used to extract regions of interest from an image for subsequent analysis, or to mask out regions of uninteresting interest to avoid unnecessary computation and improve the overall detection efficiency of the ID card anomaly detection model.

[0175] Figure 6 An exemplary flowchart of the method for using the general document anomaly detection model described in the embodiments of this specification is shown in one specific implementation.

[0176] In some specific embodiments, the image of the document to be detected, collected by the client, is input into the document anomaly detection model for foreign object detection. The model outputs a mask image that marks the abnormal area, and then restores the abnormal area to the original document image. This allows the document image to be processed according to the location of the foreign object, removing the abnormal area and thus passing the document quality compliance test.

[0177] In one embodiment of this specification, a general training device for document anomaly detection model is also proposed. Figure 7 An exemplary schematic diagram of the structure of the general document anomaly detection model training device described in the embodiments of this specification is shown in one implementation.

[0178] like Figure 7 As shown, in one embodiment, the device includes:

[0179] The sample acquisition module 30 is used to acquire sample images, wherein the sample images include abnormal document images and corresponding labeled images, and the labeled images are pre-labeled with abnormal pixels;

[0180] The segmentation module 32 is used to input the abnormal document image into the pre-trained segmentation network to classify each pixel. The classification results of each pixel include normal pixels, abnormal pixels, and unknown pixels. The classified unknown pixels are regarded as abnormal pixels to obtain the segmented image.

[0181] Training module 34 is used to estimate the penalty coefficient corresponding to each pixel in the segmented image based on the segmentation network; calculate the abandonment loss of the segmentation network for classifying each pixel based on the penalty coefficient for each pixel in the segmented image; determine the segmentation loss based on the abandonment loss; and fine-tune the segmentation network based on the segmentation loss.

[0182] In the sample acquisition module, sample images can be obtained from known large-scale datasets or acquired on demand using devices with image acquisition capabilities, and the acquired sample images are pre-annotated. Abnormal document images can include various types of foreign object occlusion, such as finger occlusion, paper occlusion, document perforation, and document damage. In the annotated image, occluded areas or areas with missing document content are displayed in a way that differs from other normal areas.

[0183] In some embodiments, the labeled image in the sample acquisition module includes a mask image. In the mask image, the position corresponding to a normal pixel is labeled as 0, and the position corresponding to an abnormal pixel is labeled as 1. The mask image is a prior art technique that can be used to extract regions of interest in an image for subsequent analysis, or to mask out regions of uninteresting interest in an image, thereby improving the overall detection efficiency of the document anomaly detection model.

[0184] In some embodiments, after the sample acquisition module acquires the sample image, it also aligns the abnormal document images in the sample image to remove background factors other than the document itself, so as to unify and standardize the input of the document anomaly detection model.

[0185] In some more specific embodiments, the sample acquisition module aligns the abnormal document images in the sample images, specifically including:

[0186] Corner point location detection for abnormal document images;

[0187] Based on the detected corner positions, affine transformations are used to align the abnormal document images.

[0188] Optionally, corner detection can employ a regression-based method, which uses an existing model to regress and predict the positions of the four corners on the document. Based on the positions of the four corners, affine transformation is used to correct and align the sample image, further standardizing the input of the document anomaly detection model and improving the model's computational efficiency.

[0189] In the segmentation module, the embodiments in this specification employ a combination of pre-training and fine-tuning to train the existing segmentation network model. Pre-training the segmentation network model on a large-scale dataset improves the generalization ability of segmentation and the ability to detect foreign objects. Fine-tuning is then performed through a foreign object detection task, which enhances the segmentation network's ability to intercept various types of abnormal occlusions while saving computational costs.

[0190] As the types of documents differ, the layout of various identity information in the document images also varies. Faced with many types of documents, there are also many types of foreign object occlusion. Different types of foreign object occlusion correspond to abnormal regions with different shapes and sizes. Therefore, the training samples cannot include all documents and all corresponding foreign object occlusion types. If the segmentation module encounters pixels that are difficult to classify, it will directly add a new category, that is, abandon the classification in the existing category and classify the pixels of this category as unknown pixels.

[0191] For foreign object occlusion types with known shape and size features, the segmentation module can detect the abnormal region and directly classify the pixels within the region as abnormal pixels. However, for foreign object occlusion types that have not appeared in the training samples, the segmentation module will encounter difficulties in classification. In order to make the segmentation results more accurate, the module will abandon the classification of such pixels in the existing category, assign a new category to the pixels, and then treat them as abnormal pixels.

[0192] By introducing the aforementioned pixel-level abstention learning (PAL) steps, the document anomaly detection model can detect more types of foreign object occlusion, including known foreign object occlusion types contained in the training samples and unknown foreign object occlusion types. The detection accuracy for unknown foreign object occlusion types is improved. At the same time, it can reduce the risk of classification errors in the document anomaly detection model.

[0193] In the training module, the penalty coefficient is used to evaluate the accuracy of the segmentation network in classifying each pixel, specifically to evaluate whether the segmentation module has classified pixels that should have been classified as normal or abnormal pixels as unknown pixels. The "penalty" represents the penalty for abandoning the classification of a pixel and assigning it as an unknown pixel; therefore, the penalty coefficient for normal or abnormal pixels is higher than that for unknown pixels, meaning the penalty coefficient for pixels other than unknown pixels is higher than that for unknown pixels.

[0194] In some embodiments, the training module estimates the penalty coefficient corresponding to each pixel in the segmented image based on the segmentation network, including:

[0195] The segmentation network defines an energy function to determine the energy of each pixel in the segmented image. The energy is used to comprehensively represent the probability that each pixel belongs to different categories. The energy of unknown pixels is higher than that of other pixels.

[0196] For each pixel in the segmented image, the corresponding penalty coefficient is estimated based on its energy.

[0197] By establishing an Energy-based Model (EBM), an energy function based on a segmentation network is constructed, combining the segmentation network model with the energy model. Each pixel in the document image is assigned a scalar called energy. The energy of each pixel is obtained by exponentially summing the probabilities of that pixel belonging to different categories. The energy of each pixel is negatively correlated with this exponential sum, and the energy is a negative number. Since the probability of an unknown pixel belonging to a newly added unknown pixel category is much greater than its probability of belonging to other existing categories, and the probability of normal or abnormal pixels belonging to existing categories is much greater than their probability of belonging to unknown pixels, the energy of unknown pixels is higher.

[0198] In some more specific embodiments, the training module calculates a penalty coefficient for each pixel in the segmented image based on the square of its energy. Since energy is a negative number, the penalty coefficient for each pixel is negatively correlated with its energy. After calculating the energy of each pixel using the EBM model, the penalty coefficient for each pixel is then calculated. Normal or abnormal pixels have lower energy and higher penalty coefficients, indicating that they should be classified into an existing category; unknown pixels have higher energy and lower penalty coefficients, indicating that they should not be classified into an existing category.

[0199] The initial PAL model normalizes unknown classifications for all pixels using a predefined fixed penalty coefficient. However, in complex scenes, different penalties are often required for different pixels. By combining PAL with EBM, an adaptive energy-based dynamic penalty coefficient can be estimated. This penalty estimation can adaptively learn an energy-based penalty to adapt to the diverse pixel features in the scene, thereby achieving accurate detection of different types of foreign objects.

[0200] In some more specific embodiments, the training module, for each pixel in the segmented image, estimates the corresponding penalty coefficient based on its energy and then sets it as follows:

[0201] Set an energy threshold, and calculate the energy loss based on the energy threshold and the energy corresponding to each pixel in the segmented image.

[0202] For each pixel in the segmented image, the abandonment loss of the segmentation network for classifying each pixel is calculated based on the penalty coefficient. The abandonment loss is negatively correlated with the probability of classifying the pixel as an unknown pixel or as another pixel.

[0203] The partition loss is calculated based on energy loss and abandonment loss;

[0204] The segmentation network is fine-tuned with the goal of minimizing the segmentation loss.

[0205] The energy threshold is used to compare the energy of each pixel. By minimizing energy loss, the energy of each pixel can be controlled within a specific range during training, so that pixels of different categories have different energy ranges.

[0206] In some more specific embodiments, the penalty coefficient exists as the denominator in the abandonment loss, while the numerator represents the probability of classifying a pixel as an unknown pixel. The probability of classifying a pixel as an existing category is added to the former in a one-term form. Under the training objective of minimizing the abandonment loss, the penalty coefficient is larger for normal or abnormal pixels, so maximizing the probability of classifying a pixel as an existing category is more efficient as the training objective; for unknown pixels, the penalty coefficient is smaller, so maximizing the probability of classifying a pixel as an unknown pixel, which is the numerator, is more efficient.

[0207] The corresponding segmentation loss is calculated by weighted summation of energy loss and abandonment loss, and the entire segmentation network is fine-tuned based on this segmentation loss.

[0208] In some more specific embodiments, the training module sets an energy threshold, and calculates the energy loss based on the energy threshold and the energy corresponding to each pixel in the segmented image. Specifically, this includes:

[0209] The training module sets a first energy threshold and calculates a first energy loss based on the first energy threshold and the energy of unknown pixels in the segmented image.

[0210] A second energy threshold is set, and a second energy loss is calculated based on the second energy threshold and the energy of pixels other than unknown pixels in the segmented image. The second energy threshold is lower than the first energy threshold.

[0211] The first energy threshold is used to set a lower limit for the energy of unknown pixels, and the second energy threshold is used to set an upper limit for the energy of normal or abnormal pixels. During the training process, the training module reduces the gap between the pixel and the corresponding energy threshold by minimizing the first and second energy losses, thereby widening the energy gap between normal or abnormal pixels and unknown pixels. Through training, the energy of the pixel can more accurately represent the category of the pixel.

[0212] In some embodiments, the training module calculates the abandonment loss for the segmentation network to classify each pixel based on the penalty coefficient, including:

[0213] The abandonment loss for classifying unknown pixels by the segmentation network is calculated based on the penalty coefficient and used as the first abandonment loss.

[0214] The abandonment loss for classifying pixels other than unknown pixels by the segmentation network is calculated based on the penalty coefficient, and is used as the second abandonment loss.

[0215] The abandonment loss is used to evaluate the accuracy of the segmentation network in classifying each pixel. By using adaptive penalty parameters, the probability of classifying a pixel into a specific category that needs to be maximized can be dynamically selected, thereby accurately detecting different types of abnormal objects.

[0216] In one embodiment of this specification, a general device for using a document anomaly detection model is also proposed. Figure 8 An exemplary schematic diagram of the device used in one embodiment of the general document anomaly detection model described in this specification is shown.

[0217] like Figure 8 As shown, in one embodiment, the device includes:

[0218] Image acquisition module 40 is used to acquire images of the document to be inspected;

[0219] The detection module 42 is used to input the document image into a pre-trained document anomaly detection model, detect abnormal regions in the document image and label them. The document anomaly detection model is trained using the general document anomaly detection model training method described in the embodiments of this specification.

[0220] In the image acquisition module, images can be acquired using devices with image acquisition capabilities such as smartphones, smartwatches, and cameras. The image acquisition module and the server can transmit images directly over the network, or images can be acquired using sub-devices connected to the host machine of the server via cables, such as cameras.

[0221] In some embodiments, after the image acquisition module acquires the image of the document to be detected, it also aligns the document image to remove background factors other than the document itself, thereby unifying and standardizing the input of the document anomaly detection model.

[0222] In some more specific embodiments, the image acquisition module aligns the document image, specifically including:

[0223] Corner point detection in document images;

[0224] Based on the detected corner positions, the document image is aligned using affine transformation.

[0225] Optionally, corner detection can employ a regression-based method, which trains an existing model to regress and predict the positions of the four corners on the document. Based on the positions of the four corners, affine transformation is used to correct and align the document image, further standardizing the input of the document anomaly detection model and improving the model's computational efficiency.

[0226] The detection module determines whether a pixel is normal or abnormal by calculating the energy of each pixel. During the training of the document anomaly detection model, by minimizing energy loss, abandonment loss and regularization loss, the energy difference between normal and abnormal pixels is widened, and the energy of normal pixels is significantly lower than that of abnormal pixels. Therefore, abnormal pixels can be detected based on the energy level.

[0227] In some embodiments, the method by which the detection module detects and labels abnormal regions in the document image includes labeling using a mask. In the output mask image of the same size as the original document image, the positions corresponding to normal pixels are labeled as 0, and the positions corresponding to abnormal pixels are labeled as 1.

[0228] One embodiment of this specification also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the method described in any one of the above descriptions.

[0229] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0230] It should be noted that the above examples are merely specific embodiments of the present invention, and the present invention is obviously not limited to the above embodiments, with many similar variations. All modifications that can be directly derived or conceived by those skilled in the art from the content disclosed in this invention should fall within the protection scope of this invention.

Claims

1. A general method for training a document anomaly detection model, wherein the document anomaly detection model includes a segmentation network, and the method includes the following steps: Obtain sample images, where, The sample images include abnormal document images and corresponding labeled images, wherein abnormal pixels are pre-labeled in the labeled images; The abnormal document image is input into a pre-trained segmentation network to classify each pixel. The classification results include normal pixels, abnormal pixels, and unknown pixels. Each unknown pixel identified in the classification is treated as an anomaly pixel, resulting in a segmented image; The segmentation network defines an energy function to determine the energy corresponding to each pixel in the segmented image. The energy is used to comprehensively represent the probability that each pixel belongs to different categories. The energy of unknown pixels is higher than that of other pixels. For each pixel in the segmented image, the corresponding penalty coefficient is estimated based on its energy; Set an energy threshold, and calculate the energy loss based on the energy threshold and the energy corresponding to each pixel in the segmented image; For each pixel in the segmented image, the abandonment loss of the segmentation network for classifying each pixel is calculated according to the penalty coefficient, wherein the abandonment loss is negatively correlated with the probability of classifying the pixel as an unknown pixel or the probability of classifying it as another pixel; The partition loss is calculated based on the energy loss and the abandonment loss; The segmentation network is fine-tuned with the goal of minimizing the segmentation loss.

2. The training method for the general document anomaly detection model as described in claim 1, wherein an energy threshold is set, and the energy loss is calculated based on the energy threshold and the energy corresponding to each pixel in the segmented image, including: Set a first energy threshold, and calculate a first energy loss based on the first energy threshold and the energy of unknown pixels in the segmented image; A second energy threshold is set, and a second energy loss is calculated based on the second energy threshold and the energy of pixels other than unknown pixels in the segmented image. The second energy threshold is lower than the first energy threshold.

3. The training method for the general document anomaly detection model as described in claim 1, wherein calculating the abandonment loss of the segmentation network for classifying each pixel based on the penalty coefficient includes: The abandonment loss of the segmentation network for classifying unknown pixels is calculated based on the penalty coefficient, and is used as the first abandonment loss. The abandonment loss for the segmentation network to classify pixels other than unknown pixels is calculated based on the penalty coefficient and used as the second abandonment loss.

4. The training method for the general document anomaly detection model as described in claim 1, wherein the labeled image includes a mask image.

5. The training method for the general document anomaly detection model as described in claim 1, after acquiring the sample images, further includes aligning the abnormal document images in the sample images.

6. The training method for the general document anomaly detection model as described in claim 5, specifically aligning the abnormal document images in the sample images includes: Corner point location detection is performed on the abnormal document image; Based on the detected corner positions, the abnormal document images are aligned using affine transformation.

7. A method for using a general document anomaly detection model, the method comprising the following steps: The client captures images of the documents to be inspected. The document image is input into a pre-trained document anomaly detection model to detect and label abnormal regions in the document image. The document anomaly detection model is trained using the method described in any one of claims 1 to 6.

8. A general training device for an anomaly detection model of identification documents, comprising: The sample acquisition module is used to acquire sample images, wherein the sample images include abnormal document images and corresponding labeled images, and the labeled images are pre-labeled with abnormal pixels; The segmentation module is used to input the abnormal document image into a pre-trained segmentation network to classify each pixel. The classification results of each pixel include normal pixels, abnormal pixels, and unknown pixels. The classified unknown pixels are regarded as abnormal pixels to obtain a segmented image. The training module is used to determine the energy corresponding to each pixel in the segmented image based on the energy function defined by the segmentation network. The energy is used to comprehensively represent the probability that each pixel belongs to different categories, wherein the energy of unknown pixels is higher than that of other pixels. For each pixel in the segmented image, a corresponding penalty coefficient is estimated based on its energy. An energy threshold is set, and an energy loss is calculated based on the energy threshold and the energy corresponding to each pixel in the segmented image. For each pixel in the segmented image, an abandonment loss for the segmentation network to classify each pixel is calculated based on the penalty coefficient, wherein the abandonment loss is negatively correlated with the probability of classifying the pixel as an unknown pixel or as another pixel. A segmentation loss is calculated based on the energy loss and the abandonment loss. The segmentation network is fine-tuned with the goal of minimizing the segmentation loss.

9. The training device for the general document anomaly detection model as described in claim 8, wherein the training module sets an energy threshold, and the energy loss is calculated based on the energy threshold and the energy corresponding to each pixel in the segmented image, specifically including: The training module sets a first energy threshold and calculates a first energy loss based on the first energy threshold and the energy of unknown pixels in the segmented image. A second energy threshold is set, and a second energy loss is calculated based on the second energy threshold and the energy of pixels other than unknown pixels in the segmented image. The second energy threshold is lower than the first energy threshold.

10. The training device for the general document anomaly detection model as described in claim 8, wherein the training module calculates the abandonment loss of the segmentation network for classifying unknown pixels based on the penalty coefficient, as the first abandonment loss; The abandonment loss for the segmentation network to classify pixels other than unknown pixels is calculated based on the penalty coefficient and used as the second abandonment loss.

11. The training apparatus for the general document anomaly detection model as described in claim 8, wherein the labeled image includes a mask image.

12. The training device for the general document anomaly detection model as described in claim 8, wherein after the sample acquisition module acquires the sample image, it further includes aligning the abnormal document images in the sample image.

13. The general document anomaly detection model training device as described in claim 12, wherein the sample acquisition module performs corner point position detection on the abnormal document image; Based on the detected corner positions, the abnormal document images are aligned using affine transformation.

14. A universal device for detecting document anomalies, comprising: The image acquisition module is used to acquire images of the documents to be inspected. The detection module is used to input the document image into a pre-trained document anomaly detection model, detect abnormal regions in the document image and label them, wherein the document anomaly detection model is trained using the method described in any one of claims 1 to 6.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1-6.

Images