Anti-counterfeiting detection method and device based on graph network, medium and equipment

Abstract

The embodiment of the invention discloses an anti-counterfeiting detection method based on a graph network, and the method comprises the steps: taking a region where each piece of key information in a to-be-detected certificate image collected for a certificate is located as each information region, and generating a structure graph which takes each information region as a node and takes an incidence relation between the key information in each information region as an edge; the node feature of each node in the structure diagram is determined by the information area corresponding to the node, inputting the structure diagram comprising the node features of each node into the diagram neural network model, and performing anti-counterfeiting detection on the to-be-detected certificate image through the diagram neural network model according to the input structure diagram. In a normal legal certificate, each piece of key information may have a certain association relationship, so that the method detects whether each information area in the certificate image to be detected has the association relationship or not, can accurately detect a forged certificate image, and effectively improves the security of online business based on eKYC.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of computers, and particularly relates to a graph network-based anti-counterfeiting detection method and device, a storage medium and an electronic device. BACKGROUND

[0002] With the development of the times, electronic identity authentication (eKYC) technology has been widely used in various fields.

[0003] Through the eKYC technology, a user can directly collect a certificate image of a certificate of the user himself / herself through a terminal such as a mobile phone, a tablet computer, a smart watch and the like used by the user, and upload the certificate image to a server, so that the server verifies the identity of the user according to the certificate image, which greatly facilitates the user to perform online business.

[0004] However, at the same time, with the development of image processing technology and artificial intelligence (AI) technology, attackers generate lifelike counterfeit certificates through image processing technology or artificial intelligence generated content (AIGC) technology to break through the risk of identity verification. The current anti-counterfeiting detection method is generally a pipeline scheme based on multi-task separation detection, which regards certificate tampering detection as a multi-step pipeline task. First, an image segmentation model is used to locate key areas such as a portrait, a name, and an address, and then content recognition is performed, that is, OCR recognition is performed on a text area, face detection and key point positioning are performed on a portrait area, and then rule checking and comparison are performed, such as rule checking on the OCR result and 1:1 comparison of a certificate face and a live body selfie face, and finally local counterfeit trace detection is performed, including using a special algorithm to detect whether there is a tampering trace in a specific area (such as a text background and a portrait edge). Each step of the method is independent, and information flows in one direction in the process, which cannot form feedback and global verification. For example, even if the "date of birth" recognized by the OCR is 1970, the face comparison module will only focus on whether the certificate portrait and the selfie are of the same person, and will not "question" whether the portrait looks like a person over 50 years old. Therefore, due to the lack of understanding of the internal relationship between the key areas, the method can be broken one by one.

[0005] Therefore, how to perform anti-counterfeiting detection on a certificate image uploaded by a user to identify whether the certificate image is a counterfeit certificate image is a problem to be solved. SUMMARY

[0006] Embodiments of the present specification provide a graph network-based anti-counterfeiting detection method and device, a storage medium and an electronic device to partially solve the problems in the prior art.

[0007] Embodiments of the present specification adopt the following technical solutions: The present specification provides a graph network-based anti-counterfeiting detection method, which comprises the following steps: Obtaining a to-be-detected certificate image collected for a certificate; Identifying the region where each key information in the to-be-detected certificate image is located as an information region; Taking each information region as a node of a structure graph, for each node, determining the node feature of the node according to the information region corresponding to the node in the to-be-detected certificate image; Generating a structure graph taking each information region as a node and the association relationship between the key information in each information region as an edge; Inputting the structure graph after determining the node features of each node into a pre-trained graph neural network model, and performing anti-counterfeiting detection on the to-be-detected certificate image according to the input structure graph by using the graph neural network model.

[0008] The present specification provides a graph network-based anti-counterfeiting detection device, which comprises: An acquisition module configured to acquire a to-be-detected certificate image collected for a certificate; An identification module configured to identify the region where each key information in the to-be-detected certificate image is located as an information region; A node feature determination module configured to take each information region as a node of a structure graph, for each node, determine the node feature of the node according to the information region corresponding to the node in the to-be-detected certificate image; A structure graph generation module configured to generate a structure graph taking each information region as a node and the association relationship between the key information in each information region as an edge; A detection module configured to input the structure graph after determining the node features of each node into a pre-trained graph neural network model, and perform anti-counterfeiting detection on the to-be-detected certificate image according to the input structure graph by using the graph neural network model.

[0009] The present specification provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above-mentioned graph network-based anti-counterfeiting detection method.

[0010] The present specification provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned graph network-based anti-counterfeiting detection method when executing the program.

[0011] The computer program product provided in the specification comprises a computer program, and the computer program is executed by a processor to implement the above-mentioned anti-fake detection method based on a graph network.

[0012] The above-mentioned at least one technical solution adopted by the embodiments of the specification can achieve the following beneficial effects: The embodiments of the specification disclose an anti-fake detection method based on a graph network. The method takes the area of each key information in the to-be-detected certificate image collected for a certificate as each information area, generates a structure graph taking each information area as a node and the association relationship between the key information in each information area as an edge, determines the node feature of each node in the structure graph according to the information area corresponding to the node, inputs the structure graph comprising the node features of each node into a graph neural network model, and performs anti-fake detection on the to-be-detected certificate image according to the input structure graph by the graph neural network model. Since different key information in a normal and legal certificate may have a certain association relationship, the above-mentioned method uses a graph neural network model to detect whether the to-be-detected certificate image has such an association relationship between each information area, which can accurately detect a counterfeit certificate image and effectively improve the security of online business based on eKYC. BRIEF DESCRIPTION OF DRAWINGS

[0013] The drawings described herein are used to provide further understanding of the specification, constitute a part of the specification, the illustrative embodiments of the specification and the description thereof are used to explain the specification, and do not constitute an improper limitation on the specification. In the drawings: Figure 1 The anti-fake detection method based on a graph network provided for the embodiments of the specification is a flowchart; Figure 2 The anti-fake detection device based on a graph network provided for the embodiments of the specification is a schematic diagram; Figure 3 The structure schematic diagram of the electronic device provided for the embodiments of the specification. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical scheme and advantages of the specification clearer, the technical scheme of the specification will be described clearly and completely in combination with the specific embodiments of the specification and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the specification, not all the embodiments. Based on the embodiments in the specification, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the specification.

[0015] The technical scheme provided by each embodiment of the specification will be described in detail below in combination with the drawings.

[0016] The technical scheme provided by each embodiment of the specification will be described in detail below in combination with the drawings.Figure 1 The flowchart of the anti-counterfeiting detection method based on a graph network provided in the embodiments of the present specification specifically includes the following steps: S100: Obtain a to-be-detected certificate image collected by a user using a collection device.

[0017] In the embodiments of the present specification, the method shown in Figure 1 The device for performing certificate anti-counterfeiting detection shown in the method can be any electronic device, including a personal computer, a mobile phone, a tablet computer, a server, or a server cluster composed of multiple servers, etc. Those skilled in the art should understand that any device can be used to perform the method shown in the embodiments of the present specification. Figure 1 The anomaly detection method shown does not constitute a limitation on the present application. For the convenience of description, the following will be described by taking a server as an example.

[0018] The server first obtains a to-be-detected certificate image collected by a user using a collection device. In the embodiments of the present specification, the to-be-detected certificate image can be a normal certificate image collected by the user directly on a normal certificate, or a counterfeit certificate image collected by an attacker on a counterfeit certificate or obtained by image tampering on a normal certificate image. The collection device can include a terminal device with a built-in camera or an externally connected camera, such as a mobile phone, etc.

[0019] Whether the to-be-detected certificate image is a normal certificate image or a counterfeit certificate image, the to-be-detected certificate image can be an image uploaded by the user (or the attacker) to the server through the terminal device.

[0020] S101: Identify the area where each key information in the to-be-detected certificate image is located as an information area.

[0021] After the server obtains the to-be-detected certificate image, it can identify each key information in the to-be-detected certificate image to determine the area where each key information is located in the to-be-detected certificate image as an information area.

[0022] Specifically, the to-be-detected certificate image can be input into a pre-trained target detection model with Optical Character Recognition (OCR) function, and the key information such as text and face in the to-be-detected certificate image is identified by the target detection model, and the area where the key information is located is determined as an information area. The above target detection model includes but is not limited to YOLOv8-OCR model, which is not limited in the present specification.

[0023] S102: Take each information area as a node of a structure graph, and for each node, determine the node features of the node according to the information area corresponding to the node in the to-be-detected certificate image.

[0024] In the embodiments of this specification, since different key information in normal documents often have certain correlations, such as in a normal Chinese ID card, the ID number contains a number segment indicating the place of residence, a number segment indicating gender, and a number segment indicating date of birth, the ID number, as a key piece of information in the ID card image, is correlated with the address, gender, and date of birth, which are also key pieces of information. Therefore, this specification can quantify these correlations using nodes and edges in the structure diagram.

[0025] To generate a structural diagram representing the relationships between different key information, the nodes in the structural diagram must first be determined. Therefore, in the embodiments of this specification, after the server determines the information regions where each key information in the image of the document to be detected is located, each information region can be used as a node in the structural diagram.

[0026] For each node, the server can determine its node features based on the information region corresponding to that node in the image of the document to be detected. Specifically, the server can input the information region corresponding to that node in the image of the document to be detected into a pre-trained feature extraction model, and use this feature extraction model to extract multimodal features from the information region corresponding to that node.

[0027] The multimodal features described in this specification include at least one of the following: visual features of information regions, textual semantic features, and positional features. For ease of explanation, the following description will only use the example of multimodal features that simultaneously include visual features of information regions, textual semantic features, and positional features.

[0028] The aforementioned feature extraction models may include convolutional neural network models, OCR-based text feature extraction models, and positional encoding models. For the visual features of an information region corresponding to a node, the server can input the information region into a pre-trained convolutional neural network model to extract its visual features. For textual semantic features, the server can input the information region into a pre-trained OCR-based text feature extraction model to identify the text within the information region, and then encode the identified text to obtain the textual semantic features of the information region. For positional features, the server can input the information region and the image of the document to be detected into a pre-trained positional encoding model to determine the coordinates of the information region within the document corresponding to the image of the document to be detected, and then encode these coordinates to obtain the positional features of the information region.

[0029] After obtaining the visual feature, the text semantic feature and the position feature of the information region corresponding to a node, the server can fuse the three features into a multi-modal feature as the node feature of the node. When fusing the three features, the three features can be directly spliced to obtain the multi-modal feature, which is not limited in the specification.

[0030] S103: Generate a structure graph with each information region as a node and the association relationship between the key information in each information region as an edge.

[0031] After determining each node in the structure graph and the node feature of each node, the server can determine the relationship between the nodes. Since each node corresponds to an information region containing a key information, the relationship between the nodes is essentially the association relationship between the key information contained in the information regions. Therefore, the server can first determine the type of the certificate corresponding to the to-be-detected certificate image, then determine the association relationship between the key information set in advance for the certificate type according to the certificate type, and finally generate a structure graph with each information region as a node, and for any two nodes in the structure graph, if the key information in the information regions corresponding to the two nodes has an association relationship, an edge connecting the two nodes is added.

[0032] When setting the association relationship between the key information for a certificate type in advance, the association relationship between the key information in the normal certificate of the certificate type can be set according to the visual, text semantic and position association relationship.

[0033] For example, in the setting of the certificate type as an ID card in the above example, the key information of the ID card includes 10 kinds of key information such as portrait, name, gender, nationality, birth date, address, certificate number, issuing authority, validity period and background area, and the association relationship between the certificate number and gender, birth date and address is essentially the text semantic association relationship.

[0034] Visually, the portrait also has an association relationship with the birth date and the validity period. This is because when applying for an ID card, a recent portrait photo is usually required as the portrait on the ID card, and the start date of the validity period is close to the shooting date of the portrait photo provided by the user, so according to the birth date of the user, the age of the user when providing the portrait photo can be roughly determined, and in combination with the portrait on the ID card, it can be judged whether the visual feature of the portrait conforms to the feature of the age of the user when providing the portrait photo.

[0035] In terms of position, the name in the ID card also has an association relationship with the portrait, because in the ID card in China, the information regions where the two key information of the portrait and the name are located are very close.

[0036] Of course, the above is only an example of an ID card to illustrate why several key information on the ID card has a certain correlation in vision, text semantics and position. This correlation can be set in advance.

[0037] In addition to setting this correlation in advance, it can also be determined by a pre-trained graph structure learning model whether there is a correlation between any two information regions in the to-be-detected certificate image. Specifically, the node features of each node determined in step S102, the information regions corresponding to each node, and the certificate type of the certificate corresponding to the to-be-detected certificate image can be input into the graph structure learning model, so that the graph structure learning model determines whether there is an edge connecting between each node.

[0038] In addition, the edges between the nodes in the structure graph generated in the present specification can be edges without weights or edges with weights. The edge without weight can be defaulted to have a weight of 1, and the edge with weight can represent the closeness of the correlation between the two nodes connected by the edge through the size of the weight. The greater the weight, the closer the correlation between the two nodes.

[0039] Further, the structure graph generated in the present specification can be a normal structure graph connecting two nodes by an edge, or a hypergraph connecting nodes by a hyperedge, wherein the hyperedge refers to an edge that can connect more than three nodes. Through the hypergraph, the correlation between different key information in a more complex certificate can be displayed quantitatively.

[0040] S104: inputting the structure graph with the node features of each node determined into a pre-trained graph neural network model, and performing anti-fake detection on the to-be-detected certificate image by the graph neural network model according to the input structure graph.

[0041] After obtaining the structure graph containing the node features of each node generated in step S103, the server can input the structure graph into a pre-trained graph neural network model. The graph neural network model in the present specification can include at least one graph attention network (GAT) and one classifier. The GAT is composed of a plurality of processing layers connected in sequence.

[0042] After the server inputs the structure graph into the GAT, each processing layer in the GAT can perform the following operations: According to the node feature of each node, determine the cross-attention weight between the nodes, and for each node, according to the cross-attention weight between the neighbor nodes connected to the node and the node, aggregate the node features of the neighbor nodes into the node feature of the node, and input the aggregated node feature of the node as the node feature of the node into the next processing layer.

[0043] Specifically, for a node i in the structure graph, the processing layer can first determine a neighbor node (i.e., a first-order neighbor node directly connected by an edge) of the node i in the structure graph, denoted as node n, and then determine a cross-attention weight of the node i and the node n according to the node features of the node i and the node n, while determining a weight of an edge between the node i and the node n. Then, the processing layer can determine a weighted feature of the node n conducted to the node i through the edge according to the node feature of the node n and the weight of the edge (e.g., the product of the node feature of the node n and the weight of the edge can be determined as the weighted feature of the node n conducted to the node i through the edge), and finally use the cross-attention weight obtained by the node i querying the node n to perform attention weighting on the weighted feature of the node n conducted to the node i through the edge, and add the attention-weighted weighted feature to the node feature of the node i to obtain the node feature of the node i after the node feature of the node n is aggregated to the node i.

[0044] By aggregating the node features of each neighbor node of the node i to the node i using the method, the aggregated node feature of the node i is obtained. After obtaining the aggregated node feature of each node in the structure graph, the node feature of each node in the structure graph changes, but the connection relationship between the nodes remains unchanged, so that the structure graph with updated node features at this time can be continuously input into the next processing layer of the GAT for processing.

[0045] When the last processing layer in the GAT also obtains the structure graph with finally updated node features by the above method, the last processing layer can fuse the aggregated node features of all nodes in the structure graph at this time to obtain the graph feature corresponding to the structure graph. When fusing the node features of all nodes together, the node features of all nodes in the structure graph at this time can be pooled, including but not limited to maximum pooling, minimum pooling, or average pooling, etc. The pooled features are the graph feature corresponding to the structure graph. The graph feature represents the visual, textual semantic, and positional features of the structure graph.

[0046] After the graph feature is obtained, the server can input the graph feature into a classifier in the graph neural network model to obtain a visual semantic consistency evaluation value of the structural graph output by the classifier, and finally perform anti-counterfeiting detection on the to-be-detected certificate image according to the visual semantic consistency evaluation value. The visual semantic consistency evaluation value of the structural graph is an evaluation value for indicating whether the features of the structural graph in the visual, text semantic, and position modalities are consistent with the association relationship in the visual, text semantic, and position modalities between the preset key information for the certificate type. The higher the evaluation value is, the more consistent the features of the structural graph in the three modalities are with the association relationship in the three modalities between the preset key information for the certificate type, and the higher the probability that the to-be-detected certificate image is a normal certificate image is. Conversely, the less consistent the features of the structural graph in the three modalities are with the association relationship in the three modalities between the preset key information for the certificate type, and the higher the probability that the to-be-detected certificate image is a counterfeit certificate image is. Therefore, after the visual semantic consistency evaluation value of the structural graph is obtained, it can be determined whether the visual semantic consistency evaluation value is higher than a preset evaluation value threshold. If yes, it can be determined that the to-be-detected certificate image is a normal certificate image. Otherwise, it can be determined that the to-be-detected certificate image is a counterfeit certificate image.

[0047] As can be seen from the above method, whether the node feature of each node finally output by the GAT accurately represents the features in the visual, text semantic, and position modalities is one of the important factors for the graph neural network model to accurately identify the counterfeit certificate image. Since the node feature of a node finally output by the GAT is obtained by aggregating the node features of the neighbor nodes of the node to the node feature of the node itself through the cross-attention weight multiple times, therefore, when training the graph neural network model including the GAT and the classifier, the training of the model parameters for determining the cross-attention weight between nodes in each processing layer of the GAT is crucial.

[0048] Thus, when pre-training the graph neural network model, a training sample can be obtained, the training sample including positive samples and negative samples, the positive samples including normal certificate images, and the negative samples including counterfeit certificate images. After obtaining these training samples, for each training sample, the same method shown in steps S101-S103 is used to determine the structure graph corresponding to the training sample as a sample structure graph, and the node features of each node in the sample structure graph are determined. The sample structure graph after determining the node features of each node is input into the graph neural network model to be trained, and the visual semantic consistency evaluation value of the sample structure graph output by the graph neural network model to be trained is obtained. Then, when the training sample is a positive sample, the difference between the visual semantic consistency evaluation value of the sample structure graph and the first standard evaluation value is reduced as the training target, and the model parameters of the graph neural network model are adjusted. When the training sample is a negative sample, the difference between the visual semantic consistency evaluation value of the sample structure graph and the second standard evaluation value is reduced as the training target, and the model parameters of the graph neural network model are adjusted. The model parameters of the graph neural network model adjusted at least include the model parameters for determining the cross-attention weight between each node in each processing layer of the GAT, and of course can also include the model parameters of the classifier. The first standard evaluation value is greater than the second standard evaluation value, and the first standard evaluation value can be set to 1, and the second standard evaluation value can be set to 0.

[0049] In order to further enable the above-mentioned GAT to accurately output the aggregated node features of each node, before the above-mentioned supervised training of the graph neural network model including the GAT and the classifier, the GAT can also be pre-trained separately.

[0050] Specifically, a pre-training sample can be obtained, and the pre-training sample only includes normal certificate images, that is, the pre-training sample only has positive samples and does not include negative samples. After obtaining the pre-training sample, for each pre-training sample, the same method shown in steps S101-S103 can be used to determine the structure graph corresponding to the pre-training sample as a pre-training structure graph, and to determine the node features of each node in the pre-training structure graph. Then, a target node in the pre-training structure graph is randomly determined, and the node features of the target node are masked, that is, the node features of the target node are deleted or directly set to a default value. The pre-training structure graph and the node features of the target node after the masking are input into the GAT to be trained, and the aggregated node features of each node in the pre-training structure graph output by the GAT to be trained are obtained. Finally, according to the aggregated node features of each node in the pre-training structure graph, the node features of the target node are predicted, and the difference between the prediction result and the node features of the target node before the masking is reduced as a training target to adjust the model parameters of the GAT. That is, when the GAT is pre-trained, the node features of the target node in the pre-training structure graph can be randomly masked, and then the node features of all nodes including the target node in the pre-training structure graph are continuously aggregated by the GAT to obtain the aggregated node features of all nodes. Finally, according to the aggregated node features of all nodes, the original node features of the target node are predicted, and the GAT is pre-trained in a self-supervised manner based on the prediction task.

[0051] By the above method, the correlation between different key information in normal legal certificates in terms of vision, text logic, and position in the certificate is utilized, and the graph neural network model is used to detect whether the correlation exists between the information regions in the certificate image to be detected, so that the counterfeit certificate image can be accurately detected, and the security of online business based on eKYC is effectively improved.

[0052] The above is a multi-modal anti-counterfeiting detection method provided by an embodiment of the present specification. Based on the same idea, the present specification also provides a corresponding device, a storage medium, and an electronic device.

[0053] Figure 2 A graph network-based anti-counterfeiting detection device provided by an embodiment of the present specification is shown in the schematic diagram, and the device comprises: The acquisition module 200 is configured to acquire a certificate image to be detected collected for a certificate; The identification module 201 is configured to identify regions where each key information in the certificate image to be detected is located as information regions. The node feature determination module 202 is configured to take each information region as a node of a structure graph, and determine, for each node, a node feature of the node according to an information region corresponding to the node in the to-be-detected certificate image. The structure graph generation module 203 is configured to generate a structure graph taking each information region as a node and an association relationship between key information in each information region as an edge. The detection module 204 is configured to input the structure graph in which the node features of all nodes are determined into a pre-trained graph neural network model, and perform anti-forgery detection on the to-be-detected certificate image according to the input structure graph by using the graph neural network model.

[0054] Optionally, the structure graph generation module 203 is specifically configured to determine a certificate type corresponding to the certificate, determine an association relationship between key information pre-set for the certificate type according to the certificate type, and generate a structure graph taking each information region as a node, and add an edge connected between any two nodes in the structure graph if the key information in the information regions corresponding to the two nodes has an association relationship.

[0055] Optionally, the node feature determination module 202 is specifically configured to input the information region corresponding to the node in the to-be-detected certificate image into a pre-trained feature extraction model, extract a multi-modal feature from the information region corresponding to the node by using the feature extraction model, wherein the multi-modal feature includes at least one of a visual feature, a text semantic feature and a position feature of the information region corresponding to the node, and determine the extracted multi-modal feature as the node feature of the node.

[0056] Optionally, the graph neural network model includes a graph attention network (GAT) and a classifier, and the GAT includes a plurality of processing layers connected in sequence. The detection module 204 is specifically configured to, for each processing layer in the GAT: determine cross-attention weights between nodes by using the processing layer according to the node features of the nodes, aggregate, for each node, node features of neighbor nodes connected to the node into the node feature of the node according to the cross-attention weights between the neighbor nodes and the node, input the node feature of the node obtained after the aggregation as the node feature of the node into a next processing layer, and until the node features of all nodes aggregated by the last processing layer are obtained, fuse the node features of all nodes aggregated by the last processing layer to obtain a graph feature corresponding to the structure graph. input the graph feature into the classifier to obtain a visual semantic consistency evaluation value of the structure graph output by the classifier, and perform anti-forgery detection on the to-be-detected certificate image according to the visual semantic consistency evaluation value.

[0057] Optionally, the apparatus further comprises: The training module 205 is configured to: obtain training samples in advance, the training samples comprising positive samples and negative samples, the positive samples comprising normal certificate images, and the negative samples comprising counterfeit certificate images; for each training sample, determine a structure graph corresponding to the training sample as a sample structure graph, and determine node features of nodes in the sample structure graph; input the sample structure graph with the node features of the nodes determined into a graph neural network model to be trained, to obtain a visual semantic consistency evaluation value of the sample structure graph output by the graph neural network model; when the training sample is a positive sample, take reducing a difference between the visual semantic consistency evaluation value of the sample structure graph and a first standard evaluation value as a training target, and adjust model parameters of the graph neural network model; when the training sample is a negative sample, take reducing a difference between the visual semantic consistency evaluation value of the sample structure graph and a second standard evaluation value as a training target, and adjust model parameters of the graph neural network model, wherein the model parameters of the graph neural network model adjusted at least include model parameters of the GAT used to determine cross-attention weights between the nodes.

[0058] Optionally, the training module 205 is further configured to: before training the graph neural network model, obtain pre-training samples, the pre-training samples comprising normal certificate images; for each pre-training sample, determine a structure graph corresponding to the pre-training sample as a pre-training structure graph, and determine node features of nodes in the pre-training structure graph; randomly determine a target node in the pre-training structure graph, and perform mask processing on the node features of the target node; input the pre-training structure graph and the node features of the nodes after the mask processing on the node features of the target node into the GAT to be trained, to obtain aggregated node features of the nodes in the pre-training structure graph output by the GAT to be trained; predict the node features of the target node according to the aggregated node features of the nodes in the pre-training structure graph; and take reducing a difference between a prediction result and the node features of the target node before the mask processing as a training target, and adjust model parameters of the GAT.

[0059] The specification also provides a computer-readable storage medium, the storage medium storing a computer program, the computer program being executable by a processor to implement the above-provided graph network-based anti-counterfeiting detection method.

[0060] The specification also provides a computer program product, the computer program product containing a computer program, the computer program being executable by a processor to implement the above-provided graph network-based anti-counterfeiting detection method.

[0061] Based on Figure 1The anti-counterfeiting detection method based on the graph network shown in the specification also provides Figure 3 The structural diagram of the electronic device shown. As Figure 3 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course can also include other hardware required by the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the anti-counterfeiting detection method based on the graph network described above.

[0062] The above is only an embodiment of the specification and is not intended to limit the specification. Various changes and modifications can be made to the specification by those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the specification shall be included in the scope of the claims of the specification.

Claims

1. A graph network-based anti-counterfeiting detection method, the method comprising: Acquire the image of the document to be inspected, based on the document collection process; Identify the regions containing key information in the image of the document to be detected, and designate them as information regions. Each information region is used as a node in the structure diagram. For each node, the node features are determined based on the information region corresponding to that node in the image of the document to be detected. Generate a structured graph with each information region as a node and the relationships between key information in each information region as edges; The structure graph, after determining the node features of each node, is input into a pre-trained graph neural network model. The graph neural network model then performs anti-counterfeiting detection on the document image to be detected based on the input structure graph.

2. The method as described in claim 1, generating a structure graph with each information region as a node and the relationships between key information in each information region as edges, specifically includes: Determine the document type corresponding to the document; Based on the document type, determine the relationships between the key information that are pre-defined for the document type; Generate a structure graph with each information region as a node, and for any two nodes in the structure graph, if the key information in the information regions corresponding to the two nodes are related, then add a connecting edge between the two nodes.

3. The method as described in claim 1, wherein determining the node features based on the information region corresponding to the node in the image of the document to be detected specifically includes: The information region corresponding to the node in the image of the document to be detected is input into a pre-trained feature extraction model. Through the feature extraction model, multimodal features are extracted from the information region corresponding to the node. The multimodal features include at least one of the visual features, text semantic features, and positional features of the information region corresponding to the node. The extracted multimodal features are determined as the node features of the node.

4. The method as described in claim 1 or 3, wherein the graph neural network model includes a graph attention network (GAT) and a classifier, and the GAT includes several sequentially cascaded processing layers; The graph neural network model performs anti-counterfeiting detection on the image of the document to be detected based on the input structure graph, specifically including: For each processing layer in the GAT: This processing layer determines the cross-attention weights between nodes based on the node features of each node. For each node, based on the cross-attention weights between the node and its neighboring nodes, the node features of each neighboring node are aggregated into the node features of the current node. The aggregated node features are then input into the next processing layer as the node features of the current node. This process continues until the aggregated node features of each node are obtained from the output of the last processing layer. Finally, the aggregated node features of each node from the last processing layer are fused to obtain the graph features corresponding to the structure graph. The graph features are input into the classifier to obtain the visual semantic consistency evaluation value of the structural graph output by the classifier. Based on the visual semantic consistency evaluation value, anti-counterfeiting detection is performed on the image of the document to be detected.

5. The method as described in claim 4, specifically including: pre-training a graph neural network model, comprising: Training samples are obtained in advance, including positive samples and negative samples. The positive samples include normal ID card images, and the negative samples include counterfeit ID card images. For each training sample, determine the corresponding structure graph as the sample structure graph, and determine the node features of each node in the sample structure graph. The sample structure graph, after determining the node features of each node, is input into the graph neural network model to be trained, and the visual semantic consistency evaluation value of the sample structure graph output by the graph neural network model is obtained. When the training sample is a positive sample, the model parameters of the graph neural network model are adjusted with the training objective of reducing the difference between the visual semantic consistency evaluation value of the sample structure graph and the first standard evaluation value. When the training sample is a negative sample, the model parameters of the graph neural network model are adjusted with the training objective of reducing the difference between the visual semantic consistency evaluation value of the sample structure graph and the second standard evaluation value. The adjusted model parameters of the graph neural network model include at least the model parameters in the GAT used to determine the cross-attention weights between nodes.

6. The method of claim 5, wherein before pre-training the graph neural network model, the method further comprises: Obtain pre-training samples, which include normal ID card images; For each pre-training sample, determine the corresponding structure graph as the pre-training structure graph, and determine the node features of each node in the pre-training structure graph. Randomly determine the target node in the pre-trained structure graph, and mask the node features of the target node; The pre-trained structure graph and the node features of each node after masking the node features of the target node are input into the GAT to be trained to obtain the aggregated node features of each node in the pre-trained structure graph output by the GAT to be trained. Based on the aggregated node features of each node in the pre-trained structure graph, predict the node features of the target node; The model parameters of GAT are adjusted with the training objective of reducing the difference between the prediction result and the node features of the target node before masking.

7. A graph network-based anti-counterfeiting detection device, the device comprising: The acquisition module is used to acquire the image of the document to be detected for document collection. The recognition module is used to identify the regions where key information is located in the image of the document to be detected, and to designate each information region. The node feature determination module is used to take each information region as a node in the structure graph, and for each node, determine the node feature based on the information region corresponding to the node in the image of the document to be detected. The structure diagram generation module is used to generate a structure diagram with each information region as a node and the relationship between key information in each information region as an edge. The detection module is used to input the structure graph after determining the node features of each node into a pre-trained graph neural network model, and the graph neural network model performs anti-counterfeiting detection on the document image to be detected based on the input structure graph.

8. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images