Abnormality detection method and device, storage medium and electronic equipment
By using a feature extraction model trained through comparative learning, the model extracts field region features from document images and compares them with a normal feature library, thus solving the problem of identifying counterfeit document images and achieving efficient identification and generalization capabilities for counterfeit documents.
Patent Information
- Application Number
- CN202512058427.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-10
AI Technical Summary
Existing technologies struggle to effectively identify forged document images, especially since deep learning-based methods require large amounts of labeled data and are difficult to extend to new font types. Furthermore, traditional feature extractors are not robust and are costly.
A pre-trained feature extraction model is used to extract field region image features from the document image through a contrastive learning method, and then compared with the features of normal document images in a preset feature library to identify the abnormality of the document image.
By capturing subtle differences between document images, it accurately identifies counterfeit document images, improving recognition accuracy and generalization ability, and reducing the adaptation cost to new font types.
Smart Images
Figure CN121837772A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present specification relates to the technical field of computers, and in particular, to an anomaly detection method and device, a storage medium, and an electronic device. BACKGROUND
[0002] With the development of the times, electronic identity authentication (eKYC) technology has been widely used in various fields.
[0003] Through the eKYC technology, a user can directly collect a certificate image of a certificate of the user himself / herself through a terminal such as a mobile phone, a tablet computer, a smart watch, and the like used by the user, and upload the certificate image to a server, so that the server verifies the identity of the user according to the certificate image, which greatly facilitates the user to perform online business.
[0004] However, at the same time, the risk of attackers breaking through identity verification by forging certificates is also increasing. Generally, a manually designed feature such as a SIFT feature, a HOG feature, and the like can be relied on to perform anomaly detection on a certificate image uploaded by a user. However, this method is difficult to capture subtle differences between fonts, has poor robustness, has limited generalization ability, and needs to design a feature extractor separately for each type of certificate, which is costly. The method of image classification based on deep learning, i.e., converting a font recognition problem into an image classification problem, training a classifier to recognize different font types, and performing anomaly detection on a certificate image uploaded by a user through the font types, needs a large amount of labeled data, and is difficult to extend to new font types.
[0005] Therefore, how to perform anomaly detection on a certificate image uploaded by a user to identify whether the certificate image is a forged certificate image is a problem to be solved. SUMMARY
[0006] Embodiments of the present specification provide an anomaly detection method, device, storage medium, and electronic device to partially solve the problems existing in the prior art.
[0007] Embodiments of the present specification adopt the following technical solutions: The anomaly detection method provided by the present specification comprises: obtaining a to-be-detected certificate image; determining each field region in the to-be-detected certificate image; for each field region, extracting an image feature of the field region in the to-be-detected certificate image by using a pre-trained feature extraction model; the feature extraction model is obtained by pre-training using a contrast learning method; Anomaly detection is performed on the document image to be detected based on the image features extracted from each field region of the document image to be detected, and the image features of normal document images stored in a preset feature library.
[0008] This specification provides an anomaly detection method, the method comprising: Sample pairs are pre-acquired, including positive sample pairs and negative sample pairs; the positive sample pairs include the same field region in different normal document images of the same type; the negative sample pairs include at least one of the following: field regions in normal document images of different types, different field regions in normal document images of the same type, and the same field region in normal document images and abnormal document images of the same type. The sample pair is input into the feature extraction model to be trained, so as to extract the image features of the input sample pair through the feature extraction model to be trained; When the input sample pair is a positive sample pair, the feature extraction model to be trained is trained with the goal of increasing the similarity between the image features extracted from each field region in the positive sample pair. When the input sample pair is a negative sample pair, the feature extraction model to be trained is trained with the goal of reducing the similarity between the image features extracted from each field region in the negative sample pair. The trained feature extraction model is used to extract image features from each field region of the document image to be detected. The image features extracted from each field region of the document image to be detected are used to perform anomaly detection on the document image to be detected.
[0009] This specification provides an anomaly detection device, the device comprising: The acquisition module is used to acquire images of the documents to be inspected. The determination module is used to determine each field region in the image of the document to be detected; The feature extraction module is used to extract image features of each field region from the image of the document to be detected using a pre-trained feature extraction model; the feature extraction model is trained in advance using a contrastive learning method. The detection module is used to perform anomaly detection on the document image to be detected based on the image features extracted from each field region of the document image to be detected and the image features of normal document images stored in a preset feature library.
[0010] This specification provides an anomaly detection device, the device comprising: The acquisition module is used to pre-acquire sample pairs, which include positive sample pairs and negative sample pairs; the positive sample pairs include the same field region in different normal document images of the same type; the negative sample pairs include at least one of the following: field regions in normal document images of different types, different field regions in normal document images of the same type, and the same field region in normal document images and abnormal document images of the same type. The input module is used to input the sample pair into the feature extraction model to be trained, so as to extract the image features of the input sample pair through the feature extraction model to be trained; The training module is used to train the feature extraction model to be trained when the input sample pair is a positive sample pair, with the goal of increasing the similarity between image features extracted from each field region of the positive sample pair; and to train the feature extraction model to be trained when the input sample pair is a negative sample pair, with the goal of reducing the similarity between image features extracted from each field region of the negative sample pair. The trained feature extraction model is used to extract image features from each field region of the document image to be detected. The image features extracted from each field region of the document image to be detected are used to perform anomaly detection on the document image to be detected.
[0011] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described anomaly detection method.
[0012] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described anomaly detection method.
[0013] This specification provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described anomaly detection method.
[0014] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: This specification discloses an anomaly detection method. This method uses a feature extraction model pre-trained using contrastive learning to extract image features from each field region of the document image to be detected. Based on the extracted image features and image features of normal document images stored in a pre-prepared feature library, anomaly detection is performed on the document image to be detected. This method can accurately identify whether a normal image to be detected is a forged or abnormal image by comparing the image features of field regions that attackers typically tamper with when forging documents with the image features of field regions of normal document images stored in the feature library, thereby ensuring the security of online business operations. Attached Figure Description
[0015] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart of the anomaly detection method provided in the embodiments of this specification; Figure 2 This is a schematic diagram of an anomaly detection device provided in an embodiment of this specification; Figure 3 This is a schematic diagram of another anomaly detection device provided in the embodiments of this specification; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this specification. Detailed Implementation
[0016] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.
[0017] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.
[0018] Figure 1 The flowchart of the anomaly detection method provided in the embodiments of this specification specifically includes the following steps: S100: Obtain the image of the document to be inspected.
[0019] In the embodiments described in this specification, the following are employed: Figure 1The device used for anomaly detection as illustrated can be any electronic device, including personal computers, mobile phones, tablets, servers, or server clusters consisting of multiple servers. Those skilled in the art should understand that the specific device used for execution... Figure 1 The anomaly detection method shown does not constitute a limitation of this application. For ease of description, the following description uses only a server as an example.
[0020] The server first acquires the image of the document to be detected. In the embodiments of this specification, the image of the document to be detected may be a normal document image directly captured by the user, or it may be an abnormal document image captured by an attacker from a forged document, or an abnormal document image obtained by tampering with a normal document image.
[0021] Regardless of whether the document image to be detected is a normal or abnormal document image, it can be an image uploaded to the server by the user (or attacker).
[0022] S101: Determine each field region in the image of the document to be detected.
[0023] For any document, there are usually several fields that record different types of information. For example, a Chinese ID card has at least a field for recording the name, a field for recording the ID number, and a field for recording the address. The text in these fields that record different types of information usually differs, such as in font, font size, etc. In other words, the text in different fields on the same type of document will differ, and the text in fields on different types of documents will differ even more. The embodiments in this specification will subsequently utilize these differences to identify whether the image of the document to be detected is an abnormal image.
[0024] Therefore, after the server obtains the image of the document to be detected through step S100, it can determine each field area in the image of the document to be detected.
[0025] Specifically, the server can use Optical Character Recognition (OCR) technology to identify each field area in the image of the document to be inspected.
[0026] To ensure the server accurately identifies each field region in the image of the document to be inspected, the server can perform an alignment operation on the image before determining each field region. This alignment operation includes: retaining the document region within the image as the foreground region, removing other regions as the background region, and then rotating the retained foreground region to align it to a preset rectangular area.
[0027] After performing the alignment operation described above on the image of the document to be inspected, OCR technology is then used to determine each field region in the aligned image. Additionally, each field region can be normalized to a pixel value range of [0, 1].
[0028] S102: For each field region, a pre-trained feature extraction model is used to extract the image features of that field region in the image of the document to be detected; the feature extraction model is pre-trained using a contrastive learning method.
[0029] In a specific embodiment of the present invention, the feature extraction model is specifically a model for extracting the visual representation of the font in each field region, which is the image feature described in this specification. This feature extraction model can be various CNN (Convolutional Neural Network) models, such as ResNet, EfficientNet, etc. For example, the feature extraction model can use EfficientNet-B0 as the main backbone network, removing its top classification layer and replacing it with a global average pooling layer and a fully connected layer. The output dimension of the fully connected layer is set to 512, serving as the final image feature vector.
[0030] In the embodiments of this specification, for each field region in the image of the document to be detected, the server can use a feature extraction model pre-trained through contrastive learning to extract the image features of that field region. This feature extraction model can be trained using contrastive learning frameworks such as SimCLR v2 or MoCo v2.
[0031] In the process of pre-training the feature extraction model, sample pairs can be obtained first. The sample pairs described in this specification include positive sample pairs and negative sample pairs. Regardless of whether they are positive or negative sample pairs, each sample pair in this specification consists of two field regions, and each field region is one sample in the sample pair.
[0032] Positive sample pairs can include the same field region from two different normal ID card images of the same type. For example, the field region recording the name in Zhang San's ID card and Li Si's ID card (both are normal ID cards that have not been tampered with) can constitute a positive sample pair. The field region recording the ID number in these two ID cards can also constitute another positive sample pair. Such positive sample pairs can guide the feature extraction model to ignore the semantic differences in character content and focus on extracting their common, latent visual style features (e.g., features based on stroke shape, texture indices, and spatial frequency distribution). The technical effect is that it makes the internal parameters of the feature extraction model (i.e., the feature extraction capability) converge to a state where the same legal font in different instances is mapped to adjacent regions in the feature space. This process essentially constructs a compact "feature cluster" for each legal ID field in the feature space. The position and range of this feature cluster in the feature space represent a numerical benchmark of the 'normal' font style of that field.
[0033] Negative sample pairs may include at least one of the following three types of negative sample pairs.
[0034] The first type involves field regions from two different types of normal document images. These two images can be from the same user or different users. The field regions in the two images can record the same type of information or different types of information. For example, the field region recording the name in Zhang San's ID card and Zhang San's driver's license can form a negative sample pair. Similarly, the field region recording the name in Zhang San's ID card and the field region recording the ID number in Li Si's driver's license can also form a negative sample pair.
[0035] The second type involves different field regions within two images of the same type of normal identification document. These two images can be images of the same normal identification document from the same user, or images of the same type of normal identification document from different users. For example, the field region recording the name in Zhang San's ID card and the field region recording the name in Li Si's ID card can form a negative sample pair. Similarly, the field region recording the name in Zhang San's ID card and the field region recording the ID number in Zhang San's ID card can also form a negative sample pair.
[0036] The third type involves the same field area in both normal and abnormal ID image images of the same type. This same field area is the altered field area in the abnormal ID image. For example, the field area recording the name in Zhang San's normal ID card and the field area recording the name in Zhang San's altered abnormal ID card can constitute the third type of negative sample pair. In the altered abnormal ID card, the field area recording the name is the altered field area, such as changing Zhang San's name to Zhang Si.
[0037] Both positive and negative sample pairs can be appropriately scrambled, such as by adding noise, without affecting the properties of the positive and negative sample pairs. Such negative sample pairs are used to enhance the model's robustness to macroscopic font style changes.
[0038] After obtaining the positive and negative sample pairs, a sample pair can be input into the feature extraction model to be trained, so that the model can extract image features from the input sample pair. When the input sample pair is a positive sample pair, the feature extraction model is trained with the goal of increasing the similarity between the image features extracted from each sample in the positive sample pair, so that the image features of each sample in the positive sample pair extracted by the trained model are as close as possible in the feature space corresponding to the feature extraction model. When the input sample pair is a negative sample pair, the feature extraction model is trained with the goal of reducing the similarity between the image features extracted from each sample in the negative sample pair, so that the image features of each sample in the negative sample pair extracted by the trained model are as far apart as possible in the feature space corresponding to the feature extraction model. Specifically, a contrastive loss function (including the InfoNCE loss function) can be used to determine the loss value. The similarity between the image features extracted by the feature extraction model from each sample in a positive sample pair is negatively correlated with the loss value determined by the contrastive loss function, while the similarity between the image features extracted from each sample in a negative sample pair is positively correlated with the loss value determined by the contrastive loss function. Thus, the model parameters of the feature extraction model to be trained are adjusted with the goal of minimizing the determined loss value until the feature extraction model converges, resulting in the trained feature extraction model.
[0039] During the training process described above, the Adam optimizer can be used, with an initial learning rate of 0.03, a batch size of 512, a training epoch of 300, and a cosine annealing scheduler used to adjust the learning rate. The temperature parameter τ is set to 0.1. Those skilled in the art should understand that the above training parameters are merely illustrative and are not intended to be limiting.
[0040] In step S102, the server can extract image features from each field region in the image of the document to be detected using the feature extraction model trained above.
[0041] It should be noted that the equipment used to train the feature extraction model and the methods employed... Figure 1 The devices used for anomaly detection as shown in this specification can be the same device or different devices; this specification does not impose any restrictions on this.
[0042] The feature extraction model trained as described in this manual, as a deep neural network trained for a specific target, acts as a style feature encoder. Its technical process involves receiving a standardized image patch as input, performing multiple nonlinear transformations, and outputting a low-dimensional, dense feature vector. It encodes the abstract, qualitative concept of 'style similarity' in fonts into a concrete, mathematically computable problem of distance between vectors. In the feature space, the Euclidean distance or cosine similarity between vectors becomes a quantitative proxy for font style consistency. This representation simplifies downstream anomaly detection tasks from complex pattern recognition problems to efficient nearest neighbor search problems.
[0043] S103: Based on the image features extracted from each field region of the document image to be detected, and the image features of normal document images stored in the preset feature library, perform anomaly detection on the document image to be detected.
[0044] In the embodiments of this specification, after extracting the corresponding image features from each field region of the document image to be detected in step S102, anomaly detection can be performed on the document image to be detected based on the extracted image features and the image features of normal document images pre-stored in the feature library. The anomaly detection methods differ depending on the image features of the normal document images pre-stored in the feature library. Specifically, in this specification, the server can use at least one of the following two methods to perform anomaly detection on the document image to be detected.
[0045] The first approach is to pre-obtain at least one normal document image for each type of document when constructing the feature library. Then, for each normal document image of that type, the feature extraction model trained above is used to extract image features from each field region of the normal document image. Each image feature extracted from each field region of each normal document image can then be directly stored in the feature library. The image features, their corresponding field regions, and the document type should also be stored in the feature library.
[0046] To reduce the number of image features stored in the feature library, thereby alleviating storage pressure and accelerating subsequent anomaly detection, this embodiment of the specification can further extract image features from each field region of each normal document image of this type, and then cluster all extracted image features based on the similarity between the extracted image features to obtain image feature clusters. The DBSCAN clustering method can be used for clustering, or other clustering methods can be used; this specification does not impose any restrictions on this.
[0047] After clustering, for each image feature cluster, at least one field region corresponding to that cluster can be determined. Then, based on the image features contained within that cluster, a representative image feature is determined. Finally, the representative image features determined for each cluster, the field regions corresponding to the image feature clusters containing those representative features, and the document type are stored in a feature database. Specifically, when determining the field region corresponding to an image feature cluster, for each field region, the image feature cluster containing the most image features extracted from each normal document image within each cluster is selected as the image feature cluster corresponding to that field region. When determining the representative image feature of an image feature cluster, the cluster center of that cluster is determined based on each image feature contained within it, and this cluster center is used as the representative image feature of that cluster.
[0048] After constructing the feature library using the above method, in step S103, when performing anomaly detection on the document image to be detected, the server can extract image features from each field region in the document image to be detected as image features to be detected. It can also extract representative image features from the preset feature library corresponding to the document type and the field region of the document image to be detected as comparison image features. Then, it determines the similarity between the image features to be detected and the comparison image features. Finally, based on the similarity determined for each image feature to be detected, anomaly detection is performed on the document image to be detected. Specifically, if the similarity between at least one image feature to be detected and the corresponding comparison image feature is lower than a preset threshold, the document image to be detected is determined to be an abnormal image; if the similarity between all image features to be detected and the corresponding comparison image features is not lower than the preset threshold, the document image to be detected is determined to be a normal image.
[0049] The second approach involves pre-collecting at least one normal document image for each type of document. For each normal document image, a field region is identified. Then, the trained feature extraction model is used to determine the image features of each image patch within that field region. These features are then fused to obtain the image features of that field region within the normal document image. Specifically, when determining the image features of each image patch within a field region, a sliding window of a preset size can be used. This window slides across the field region with a preset step size, and each slide collects a portion of the image within the sliding window as an image patch. This patch is then input into the feature extraction model to obtain its image features. When fusing the image features of each image patch within a field region, the image features of each patch can be pooled, and the pooled features are used as the image features of the field region. Pooling methods include max pooling, min pooling, or average pooling.
[0050] After obtaining the image features of each field region in each normal document image, representative image features for that field region of a normal document image of that type can be determined based on the determined image features of that field region in each normal document image. Specifically, coreset sampling or principal component analysis (PCA) can be used to determine the representative image features of the same field region in normal document images of that type. Finally, each representative image feature and the corresponding document type are stored in a feature library. The feature library obtained using this method is the PatchCore in-memory library.
[0051] After constructing the feature library using the above method, in step S103, when performing anomaly detection on the document image to be detected, the server can extract the image features extracted from each field region in the document image to be detected as the image features to be detected, extract all representative image features corresponding to the type of document corresponding to the document image to be detected from the preset feature library, and then select a specified number of representative image features from each extracted representative image feature in descending order of similarity with the image features to be detected as comparison image features. Based on the similarity between the image features to be detected and each selected comparison image feature, the anomaly representation value of the image features to be detected is determined, and anomaly detection is performed on the document image to be detected based on the anomaly representation value determined for each image feature to be detected.
[0052] The reason why the second method mentioned above does not need to store the field regions corresponding to each representative image feature when constructing the feature library, but only needs to store the type of certificate corresponding to each representative image feature, is because the representative image features stored in the feature library at this time are the normal image features of each field region of all normal certificate images of this type, or in other words, all local image features of all normal certificate images of this type. It is equivalent to the feature library having "memorized" all local image features of normal certificate images of this type.
[0053] When an attacker tampers with an image, the abnormal area (i.e., the area tampered with) can appear in any field region. Therefore, the anomaly is not related to the location of the field region in the image to be detected. Thus, during anomaly detection, for each field region in the image to be detected, a nearest neighbor search method can be used. Based on the image features to be detected for that field region, a specified number of representative image features with the highest similarity to the image features to be detected are searched from all representative image features of that type in the feature library. If the image features to be detected are not close to these representative image features, it means that the field region in the image to be detected does not match the "memory" of the feature library, and the image to be detected is an abnormal image. Therefore, the server can determine the anomaly representation value of the image features to be detected based on the similarity between the image features to be detected and each selected comparison image feature. The higher the similarity between the image features to be detected and each selected comparison image feature, the lower the anomaly representation value of the image features to be detected, and vice versa. If at least one field region of the image to be tested has an abnormal feature value exceeding a preset threshold, the image to be tested can be determined to be an abnormal image. If none of the field regions of the image to be tested have an abnormal feature value exceeding the preset threshold, the image to be tested can be determined to be a normal image.
[0054] Of course, the two anomaly detection methods mentioned above can be combined. Anomaly scores can be determined by the two methods respectively, and a weighted anomaly score can be determined based on the weights set for the two methods. If the weighted anomaly score exceeds the preset threshold, the image to be detected is determined to be an abnormal image; otherwise, the image to be detected is determined to be a normal image. This will not be elaborated on here.
[0055] In summary, the method described above, through training the internal representation capabilities of the feature extraction model, can capture subtle differences between fonts on different documents. Moreover, it eliminates the need to design and train a separate feature extraction model for each document type; only a general feature extraction model for extracting image features needs to be trained. This method is also applicable to new fonts and has strong generalization ability.
[0056] The above is an example of an anomaly detection method provided in this specification. Based on the same idea, this specification also provides corresponding devices, storage media, and electronic devices.
[0057] Figure 2 This is a schematic diagram of an anomaly detection device provided in an embodiment of this specification. The device includes: The acquisition module 200 is used to acquire the image of the document to be detected; The determining module 201 is used to determine each field region in the image of the document to be detected; Feature extraction module 202 is used to extract image features of the field region in the image of the document to be detected by using a pre-trained feature extraction model for each field region; the feature extraction model is trained in advance using a contrastive learning method; The detection module 203 is used to perform anomaly detection on the document image to be detected based on the image features extracted from each field region of the document image to be detected and the image features of normal document images stored in a preset feature library.
[0058] Optionally, the detection module 203 is further configured to: acquire at least one normal document image for each type of document in advance; extract image features from each field region of the normal document image using the feature extraction model for each normal document image of that type; cluster the image features extracted from each field region of each normal document image of that type to obtain each image feature cluster; determine at least one field region corresponding to each image feature cluster, and determine the representative image features of the image feature cluster based on each image feature contained in the image feature cluster; and store each representative image feature, the field region corresponding to the image feature cluster in which each representative image feature is located, and the type of document in a feature library.
[0059] Optionally, the detection module 203 is specifically configured to: for each field region in the image of the document to be detected, extract image features from that field region as image features to be detected, and extract representative image features corresponding to the document type and the field region from a preset feature library as comparison image features; determine the similarity between the image features to be detected and the comparison image features; and perform anomaly detection on the image of the document to be detected based on the similarity determined for each image feature to be detected.
[0060] Optionally, the detection module 203 is further configured to: acquire at least one normal document image of each type of document in advance; determine each field region in each normal document image of the type; for each field region in the normal document image, use the feature extraction model to determine the image features of each image block in the field region, fuse the image features of each image block in the field region to obtain the image features of the field region in the normal document image; for each field region, determine the representative image features of the field region of the normal document image of the type based on the determined image features of the field region in each normal document image; and store each representative image feature and the type of document corresponding to each representative image feature in a feature library.
[0061] Optionally, the detection module 203 is specifically configured to: for each field region in the image of the document to be detected, extract image features from that field region as image features to be detected; extract all representative image features corresponding to the type of document corresponding to the image of the document to be detected from a preset feature library; select a specified number of representative image features from the extracted representative image features in descending order of similarity to the image features to be detected, as comparison image features; determine the abnormal characterization value of the image features to be detected based on the similarity between the image features to be detected and each selected comparison image feature; and perform anomaly detection on the image of the document to be detected based on the abnormal characterization value determined for each image feature to be detected.
[0062] Figure 3 This is a schematic diagram of another anomaly detection device provided in the embodiments of this specification. The device includes: The acquisition module 300 is used to pre-acquire sample pairs, which include positive sample pairs and negative sample pairs; the positive sample pairs include the same field region in different normal document images of the same type; the negative sample pairs include at least one of the following: field regions in normal document images of different types, different field regions in normal document images of the same type, and the same field region in normal document images and abnormal document images of the same type. The input module 301 is used to input the sample pair into the feature extraction model to be trained, so as to extract the image features of the input sample pair through the feature extraction model to be trained; The training module 302 is used to train the feature extraction model to be trained when the input sample pair is a positive sample pair, with the goal of increasing the similarity between image features extracted from each field region of the positive sample pair, and to train the feature extraction model to be trained when the input sample pair is a negative sample pair, with the goal of reducing the similarity between image features extracted from each field region of the negative sample pair. The trained feature extraction model is used to extract image features from each field region of the document image to be detected. The image features extracted from each field region of the document image to be detected are used to perform anomaly detection on the document image to be detected.
[0063] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can be used to perform the above-described anomaly detection method.
[0064] This specification also provides a computer program product comprising a computer program that, when executed by a processor, implements the above-described anomaly detection method.
[0065] based on Figure 1 The anomaly detection method shown in this specification also provides embodiments that provide... Figure 4 The diagram shows the structure of the electronic device. Figure 4 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the aforementioned anomaly detection method.
[0066] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.
Claims
1. An anomaly detection method, the method comprising: Obtain the image of the document to be inspected; Determine the field region of each field in the image of the document to be detected; For each field region, a pre-trained feature extraction model is used to extract the image features of that field region in the document image to be detected; the feature extraction model is pre-trained using a contrastive learning method. Anomaly detection is performed on the document image to be detected based on the image features extracted from each field region of the document image to be detected, and the image features of normal document images stored in a preset feature library.
2. The method as described in claim 1, wherein the preset feature library specifically includes: For each type of document, obtain at least one normal document image for that type in advance; For each normal document image of this type, the feature extraction model is used to extract image features from each field region of the normal document image; Clustering is performed on the image features extracted from each field region of each normal document image of this type to obtain image feature clusters; For each image feature cluster, at least one field region corresponding to the image feature cluster is determined, and the representative image feature of the image feature cluster is determined based on each image feature contained in the image feature cluster. The feature library stores each representative image feature, the field region corresponding to the image feature cluster to which each representative image feature belongs, and the type of document.
3. The method as described in claim 2, wherein anomaly detection is performed on the document image to be detected based on image features extracted from each field region of the document image to be detected and image features of normal document images stored in a preset feature library, specifically including: For each field region in the image of the document to be detected, the image features extracted from that field region are used as the image features to be detected. In addition, the representative image features corresponding to the type of document and the field region of the document to be detected are extracted from the preset feature library as the comparison image features. Determine the similarity between the features of the image to be detected and the features of the comparison image; Anomaly detection is performed on the document image to be detected based on the similarity determined for each feature of the image to be detected.
4. The method as described in claim 1, wherein the preset feature library specifically includes: For each type of document, obtain at least one normal document image for that type in advance; For each normal document image of this type, determine each field region in the normal document image; For each field region in the normal document image, the feature extraction model is used to determine the image features of each image block in the field region, and the image features of each image block in the field region are fused to obtain the image features of the field region in the normal document image; For each field region, based on the image features of that field region in each normal document image, determine the representative image features of that field region for that type of normal document image; The feature database stores the representative image features and the corresponding document types.
5. The method as described in claim 4, wherein anomaly detection is performed on the document image to be detected based on image features extracted from each field region of the document image to be detected and image features of normal document images stored in a preset feature library, specifically including: For each field region in the image of the document to be detected, the image features extracted from that field region are used as the image features to be detected; Extract all representative image features corresponding to the type of document corresponding to the document image to be detected from the preset feature library; According to the order of similarity with the features of the image to be detected from high to low, a specified number of representative image features are selected from each of the extracted representative image features as comparison image features; Based on the similarity between the features of the image to be detected and each selected comparison image feature, the abnormal characterization value of the features of the image to be detected is determined; Anomaly detection is performed on the document image to be detected based on the anomaly representation value determined for each feature of the image to be detected.
6. An anomaly detection method, the method comprising: Sample pairs are pre-acquired, including positive sample pairs and negative sample pairs; the positive sample pairs include the same field region in different normal document images of the same type; the negative sample pairs include at least one of the following: field regions in normal document images of different types, different field regions in normal document images of the same type, and the same field region in normal document images and abnormal document images of the same type. The sample pair is input into the feature extraction model to be trained, so as to extract the image features of the input sample pair through the feature extraction model to be trained; When the input sample pair is a positive sample pair, the feature extraction model to be trained is trained with the goal of increasing the similarity between the image features extracted from each field region in the positive sample pair. When the input sample pair is a negative sample pair, the feature extraction model to be trained is trained with the goal of reducing the similarity between the image features extracted from each field region in the negative sample pair. The trained feature extraction model is used to extract image features from each field region of the document image to be detected. The image features extracted from each field region of the document image to be detected are used to perform anomaly detection on the document image to be detected.
7. An anomaly detection device, the device comprising: The acquisition module is used to acquire images of the documents to be inspected. The determination module is used to determine each field region in the image of the document to be detected; The feature extraction module is used to extract image features of each field region from the image of the document to be detected using a pre-trained feature extraction model; the feature extraction model is trained in advance using a contrastive learning method. The detection module is used to perform anomaly detection on the document image to be detected based on the image features extracted from each field region of the document image to be detected and the image features of normal document images stored in a preset feature library.
8. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1-6.
10. A computer program product comprising a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.